SAP HANA RAMBLINGS


SAP HANA is a leading IMDB(In-memory DB, not the movies one:)). It works by keeping all required in RAM. The idea of developing in-memory cache was started with mounting pressure from BW customers for faster query execution times. Also the same case with APO CIF queues. SAP answered this query by developing live cache for APO and BW accelerator, both works keeping data in-memory. This prompted to develop a fully operational DB in-memory, SAP HANA. The cheaply available RAM costs are also a main reason.

SAP HANA is configured for parallel processing and distributed processing.

When a query is passed to HANA DB engine, HANA observes the generated execution plan of the query. Then it identifies the parts of query that can be computed in parallel there by reducing response time.

 Also, HANA is deployed in infrastructure that has disaster tolerance and high availability. SAP teams up with hardware vendors like HP,Hitachi, IBM to develop this hardware infrastructure. These vendors provide SMP machines i.e. 2 or 4 CPU's work by sharing same main memory. This distributed computing provides these advantages.
1. When one of them is down, still the system is available.
2. Also by adding more CPU's increases scalability.

Columnar Tables : We have seen traditional RDBMS where data is stored in rows. However these systems are better suited for transactions system where the whole record is required for processing and context is more data insert/delete/modify sensitive. For analytic/reporting  purposes we don't need all the 200 fields in sales order table. We might be interested in only net sales, tax amounts and discounts. Also in analytic systems, processing is more about at aggregate level rather than at record level. We are mostly interested in SUM,MIN, MAX values of particular aggregate. In total, we are interested in aggregated values of columns rather than the whole record. Hence traditional RDBMS is not an ideal choice for analytic systems. Because in RDBMS data is stored physically on circular discs. Each disc is composed of tracks. All the rows are arragned sequentially in tracks.

When we asked DB engine to calculate SUM of a particular column, it goes to each row and offsets to column of the row, then rotates the disc till next row and puts DB reader to column location and reads the column value. Hence the total time takes equals to rotation time till next row and reading time for corresponding column. Also as the data is stored on HDD, the reading times of ROM are way slower than reading times of main memory(RAM).

In case of column tables, data is stored column wise and compression, dictionary encoding techniques will be employed as shown below. After a customer survey report, many SAP table columns contains small number of distinct values repeated  through out the column like material colors, properties. After carefully examining may customers data, SAP came to this conclusion.  It was estimated that master data tables size could be reduced by 80% with compression technique. Also instead of processing text values, we could assign integer values for each distinct value, there by making operations like COUNT faster.
     As the column values are stored sequentially, for operations like  SUM,MIN,MAX, disc header proceeds sequentially; hence the reading times are lowered. Also as the data resides in RAM, which has higher processing speeds. Also as the column data is stored sequentially, indexes are not required.

All these factors explained above, parallel processing, distributed computing, data compression, data encoding, in-memory residence makes processing of aggregate operations very faster. Hence HANA DB is ideal for analytic operations and with lightening speeds. Your enterprise data can be searched/analysed at the speed we use Google.

Note: Although data compression, encoding takes extra processing times, overhead is very much small when  compared to row store DB processing times. SAP has mechanisms to write data changes to persistent area when there is an failure as data resides in main memory.We will see in an other post about HANA memory management.

If column  tables are efficient/prominence in HANA DB, why are there row tables:

HANA is not only a DB for analytic operations, it supports transactions systems too, hence the traditional row tables are supported. SAP vision is to run both analytic and transaction systems on one DB. Then you can do analytic operations on real time data seamlessly. Hence we can think about real time data analytics rather than working with pre-aggregated data which will upadted once in the month end only.






Comments

Post a Comment

Popular posts from this blog

SAP V1 V2 V3 JOBS

Image
Before starting with V-series jobs, lets review some fundamentals. Data base LUW:  It consists of sequence of DB operations(INSERT,UPDATE,DELETE) that needs to be committed to either all or none to keep DB in consistent state. COMMIT will be executed when all DB operations were successfully executed, once COMMITTED these changes can't be ROLLED BACK. This is part of traditional ACID property of any RDBMS. If a failure happens while executing the LUW(will be mostly technical like memory crunch,integrity violation or data base crashed,power failure) and when DB is restarted, DB will be ROLLed BACK to previous state. Which is nothing but the state when last COMMIT was happened. In other words, when ROLL BACK happens DB will be safely brought to previous COMMIT state. DB maintains log of actions performed since last COMMIT and as part of recovery process, all these actions will be reversed. All RDBMS will have in-built locking/multi concurrency control such that dirty READ-WRITE ope...
Read more

Virtualization with SAP HANA

Image
Virtualization Most of Corporate Enterprise warehouse Eco system consists multiple disparate  Data warehouses. reason can be attributed to mergers and acquisitions,  departments using specialized software, databases choose to support specific data needs. All these scenarios result in building multiple data marts over multiple technologies. However, to serve the purpose of single source of truth, we end up building Enterprise Data Warehouse(EDW)  by sourcing data from various data marts across organization. One way of doing is, running a project for establishing standard ETL methods to extract data from regional Data Marts and staging data into EDW. However various departmental/ regional DW already contains highly formatted, information rich and aggregated data built readily available. This implies that we might not need to 1. Perform extensive transformations 2. Source data is meta data rich 3. Less data volumes will be retrieved from source systems Following trad...
Read more

Star Join node Vs Analytic view && Attribute View vs Calculation Dimension View

Analytic views comes with constraint that Measures should be coming from a single of Data foundation, although data foundation could contain any number of tables joined. Previously, we used model multiple Analytic views and combine them using JOIN node of an calculation view, in order to facilitate to Measures from multiple tables.  Star Join solves this by allowing measures coming from multiple tables. So we don't have to use multiple Analytic view to achieve this.  However Analytic views comes enriched with master data through Attribute views. To compensate this, We have Calculation Dimensional Models introduced. These artifacts are same as  Attribute views, but can only be used in a Star Join. So this solves out Master data enrichment issue while using Star Join. Introduction of Star Join Node and Calculation Dimensional Models  certainly introduces more uses cases for Calculation views and avoids scenario of creating multiple Analytic views for e...
Read more