Ramblings on SAP BW Powered by HANA
Most of customer's Use-case for HANA appliance is BW on HANA Database(HDB). This is the entry point of HANA into most of the customers Landscape. Here I will quickly try to cover what's new with BW 7.3 and 7.4 on HANA.
Migrating BW onto HANA can be primarily seen from two perspectives.
1. Copying data from legacy DB to HANA
2. SAP BW application server migration onto HDB
Here I am referring to migration with minimum system downtime. We will initally copy data into HANA and when everything is ready, will take down BI system down. Then migrate application servers too onto HANA. This ensures minimum service disruption time.
Database Migration : Mass copying of data from Legacy into HDB, in general will be handled by in house BASIS team or your HANA hardware vendor can help in DB migration. In the Software Update Manager(SUM), we have Database Migration Option(DMO). This makes our job easier to copy data foot print. However also note that, due to HANA Column storage, compression and dictionary encoding, it is observed that data footprint will be reduced by 400% when copied to HANA. For example, your 52 TB data footprint might come down to 13 TB. This should be taken due care for HANA Box Sizing options as SAP charges on the basis of GB's of memory.
Application Server Migration : We will quickly digress from migration topic and will review HANA Optimized Artifacts.
In-memory Optimized(IMO) Cubes: Thanks to lightening processing speeds of HANA and powerful engines, we have moved from Extended star schema to Star Schema from a conceptual point of view. There will be no DIMID tables and Fact table will directly uses SID tables. In other words, every dimension is Line-Item Dimension now. This will greatly improve Cube loading times as there is no need to check for referential integrity and generate DIMID for a new combination of SID's. However, still we have to specify Dimensions while modelling. The will be merely used as sorting criteria.
Elimination of DIMID tables, we can relate more to real world data models to BW dimensional Modelling.
So what does this mean?
1. Line-Item dimensions and High-Cardinality follows from Kimberley Dimensional modelling principles for Degenerated Dimensions. But there wont be anymore Line Item Dimensions(as each dimension is already one) and High-cardinal options for large Dimension tables on BW powered by HANA.
2. As data is already stored in columns, no indexes will be required. The index option from DSO is taken out. This implies that updating DB statistic, Missing indexes on DB are something, we can leave out.
Hence, we can remove these steps from our process chains. Also there is no harm in keeping them; they are simply not doing anything. Also we can get rid of ACR, Aggregates as we aren't going to make up any results.
A. Attribute change run
B. Delete and Create Indexes while loading into Cube
C. Updating of DB statistics
D. Maintaining Aggregates/ Roll ups
3. In BW 7.3 In-Memory is an option, however in 7.4 it is by default; means any new cube created is IMO. However in 7.3 we can leave it as normal InfoCube(IC) or as IMO IC.
4. Although DIMID tables are eliminated, the SAP BW rule of having 16 Dimensions, 3 Technical(Package, Time, Unit) and 233 key figures still hold valid. As it is a Rule for any DB to have maximum of 255 fields and primary key can have only upto 16 fields and HDB is no exception.
5. Partitions : From partition of cubes based on time characteristics to SPO, now we don't need to consider options to partition Cube data. IMO cube by default will be having Four partitions. Each one for Uncompressed requests, compressed requests, intialization of Inventory Stocks and Historic Movements data respectively. Last two partitions are relevant when we are dealing cubes from Inventory Management (OIC_C03). Although a cube is not relevant for Inventory(Field STKUPD from ROOSOURCE table enabled for corresponding Datasources), the last two partitions will be created. However they will be doing nothing and holds empty.
In-Memory DSO
The main change for DSO comes in the form of Change log. Although the active data table and activation queue tables are still there, Change log will be computed via calculated view on these tables. This will reduce data foot print. The previously created HANA optimized DSO are obselete and must converted to In-memory Optimized DSO's using RSDRI_RECONVERT_DATASTORE.
The DSO activation process is pushed down to HDB as stored procedure. This will greatly reduce the DSO activation time. While converting DSO to IMO DSO, we have option of keeping previous change log data else could leave it. While keeping change log will longer conversion times, we can't roll back previous requests.
As now that DSO activation times lowered considerably, we can think of avoiding Info cubes where we aren't performing anymore transformations, data unification from multiple sources or in other words not changing data or modelling additional dimensions, we can safely remove Info cubes. DSO's will do the job as Aggregation operation is not intensive on HDB.
Also most of the operations supported OLAP engine, Exception Aggregation, displaying hierarchies, currency conversions will be pushed down to HDB in future.
LookUPs: As we strive to write efficient Lookups using ABAP to fill Master data by using Lookup operations, we have few more options of the task.
1. Read from DSO option is added in addition to read from master data
2. Database hints for FOR ALL ENTRIES can be used. This is specially useful when your Driver tables is large. As most of the time the query takes longer times or goes to dump if driver table contains more than 50000 rows. However to use the hint, the driver table should only contain "EQ" operation.
Note :
1. Although as mentioned above, data activation times for info providers is decreased considerably, the data loading times into PSA has not seen much decrease.
2. In the same way, SPO's can be converted to IMO and each partition becomes a IMO DSO. Also Write-Optimzed DSO are partitioned by Request ID.
The conversion of info providers should be done separately on DEV, QUA, PRD systems. They can not be transported. The conversion reports can take list of info providers for collective conversion.
Mainly there are series of SAP Notes to be implemented. These should be implemented in DEV and should be carried to production. Once, we are done with implementing required notes till quality, we can bring down BW production. The same cutover activities, we follow releases/ rollouts should be employed here(lock users/clear all delta queues(V1,V2,V3)). Also while we are converting an Info provider, no loading operations can be done, though reporting can be done.
In my next post , I will detail about Best Practices for SAP BW on HANA Testing.
Migrating BW onto HANA can be primarily seen from two perspectives.
1. Copying data from legacy DB to HANA
2. SAP BW application server migration onto HDB
Here I am referring to migration with minimum system downtime. We will initally copy data into HANA and when everything is ready, will take down BI system down. Then migrate application servers too onto HANA. This ensures minimum service disruption time.
Database Migration : Mass copying of data from Legacy into HDB, in general will be handled by in house BASIS team or your HANA hardware vendor can help in DB migration. In the Software Update Manager(SUM), we have Database Migration Option(DMO). This makes our job easier to copy data foot print. However also note that, due to HANA Column storage, compression and dictionary encoding, it is observed that data footprint will be reduced by 400% when copied to HANA. For example, your 52 TB data footprint might come down to 13 TB. This should be taken due care for HANA Box Sizing options as SAP charges on the basis of GB's of memory.
Application Server Migration : We will quickly digress from migration topic and will review HANA Optimized Artifacts.
In-memory Optimized(IMO) Cubes: Thanks to lightening processing speeds of HANA and powerful engines, we have moved from Extended star schema to Star Schema from a conceptual point of view. There will be no DIMID tables and Fact table will directly uses SID tables. In other words, every dimension is Line-Item Dimension now. This will greatly improve Cube loading times as there is no need to check for referential integrity and generate DIMID for a new combination of SID's. However, still we have to specify Dimensions while modelling. The will be merely used as sorting criteria.
Elimination of DIMID tables, we can relate more to real world data models to BW dimensional Modelling.
So what does this mean?
1. Line-Item dimensions and High-Cardinality follows from Kimberley Dimensional modelling principles for Degenerated Dimensions. But there wont be anymore Line Item Dimensions(as each dimension is already one) and High-cardinal options for large Dimension tables on BW powered by HANA.
2. As data is already stored in columns, no indexes will be required. The index option from DSO is taken out. This implies that updating DB statistic, Missing indexes on DB are something, we can leave out.
Hence, we can remove these steps from our process chains. Also there is no harm in keeping them; they are simply not doing anything. Also we can get rid of ACR, Aggregates as we aren't going to make up any results.
A. Attribute change run
B. Delete and Create Indexes while loading into Cube
C. Updating of DB statistics
D. Maintaining Aggregates/ Roll ups
3. In BW 7.3 In-Memory is an option, however in 7.4 it is by default; means any new cube created is IMO. However in 7.3 we can leave it as normal InfoCube(IC) or as IMO IC.
4. Although DIMID tables are eliminated, the SAP BW rule of having 16 Dimensions, 3 Technical(Package, Time, Unit) and 233 key figures still hold valid. As it is a Rule for any DB to have maximum of 255 fields and primary key can have only upto 16 fields and HDB is no exception.
5. Partitions : From partition of cubes based on time characteristics to SPO, now we don't need to consider options to partition Cube data. IMO cube by default will be having Four partitions. Each one for Uncompressed requests, compressed requests, intialization of Inventory Stocks and Historic Movements data respectively. Last two partitions are relevant when we are dealing cubes from Inventory Management (OIC_C03). Although a cube is not relevant for Inventory(Field STKUPD from ROOSOURCE table enabled for corresponding Datasources), the last two partitions will be created. However they will be doing nothing and holds empty.
6. Compression:
I. Here cube compression is not that much priority to improve query performance. As HANA engines are faster, we don't see much improvement after compressing requests; especially for not s large cubes.
II. However we still see a Compression option. Because, in scenarios like Inventory Management(to update reference points) or for cubes which huge amounts of data daily, compression is needed.
III. It merely moves data from one partition to another partition. But this helps in bringing down cube loading times, as we will searching a smaller partition now.
7. No E table: There wont be two fact tables. As already mentioned 2nd partition will do the job.
The main change for DSO comes in the form of Change log. Although the active data table and activation queue tables are still there, Change log will be computed via calculated view on these tables. This will reduce data foot print. The previously created HANA optimized DSO are obselete and must converted to In-memory Optimized DSO's using RSDRI_RECONVERT_DATASTORE.
The DSO activation process is pushed down to HDB as stored procedure. This will greatly reduce the DSO activation time. While converting DSO to IMO DSO, we have option of keeping previous change log data else could leave it. While keeping change log will longer conversion times, we can't roll back previous requests.
As now that DSO activation times lowered considerably, we can think of avoiding Info cubes where we aren't performing anymore transformations, data unification from multiple sources or in other words not changing data or modelling additional dimensions, we can safely remove Info cubes. DSO's will do the job as Aggregation operation is not intensive on HDB.
- IMO DSO only contains last record if same record appears multiple times in a package.
- Unique data records property isn't doing anything. As they performance gain is very low, it doesn't check for uniqueness.
- Always select "generate SID's on data activation". This will give query performance increase.
Also most of the operations supported OLAP engine, Exception Aggregation, displaying hierarchies, currency conversions will be pushed down to HDB in future.
LookUPs: As we strive to write efficient Lookups using ABAP to fill Master data by using Lookup operations, we have few more options of the task.
1. Read from DSO option is added in addition to read from master data
2. Database hints for FOR ALL ENTRIES can be used. This is specially useful when your Driver tables is large. As most of the time the query takes longer times or goes to dump if driver table contains more than 50000 rows. However to use the hint, the driver table should only contain "EQ" operation.
Note :
1. Although as mentioned above, data activation times for info providers is decreased considerably, the data loading times into PSA has not seen much decrease.
2. In the same way, SPO's can be converted to IMO and each partition becomes a IMO DSO. Also Write-Optimzed DSO are partitioned by Request ID.
The conversion of info providers should be done separately on DEV, QUA, PRD systems. They can not be transported. The conversion reports can take list of info providers for collective conversion.
Mainly there are series of SAP Notes to be implemented. These should be implemented in DEV and should be carried to production. Once, we are done with implementing required notes till quality, we can bring down BW production. The same cutover activities, we follow releases/ rollouts should be employed here(lock users/clear all delta queues(V1,V2,V3)). Also while we are converting an Info provider, no loading operations can be done, though reporting can be done.
In my next post , I will detail about Best Practices for SAP BW on HANA Testing.
Comments
Post a Comment