Overview of SAP HANA Dynamic Tiering.
SAP HANA is a unique technology, where all data is loaded and processed in-memory. As data grows, the SAP HANA hardware must grow – first in-memory capacity in a scale-up system, and then to a scale-out larger system.
But, do all enterprises require the high performance of HANA memory? Can one de-couple data growth from HANA memory growth and move some of the data to a lower-cost, lower-performing storage and processing tier?
A temperature analogy is often used to describe the value of data at a particular point in time.
Hot data refers to highly active and current data that needs to be accessed with low latency. In a HANA system, hot data should be maintained in HANA memory.
A cold store is an archive store - data is kept for possible future reference, compliance purposes, or deep historical trend analysis. This kind of data does not change and does not partake in normal administrative operations, such as regular backups. Archive stores are kept separate from the HANA database and stored on disk using a technology such as SAP IQ or Hadoop.
There is one useful additional tier that is distinguished from hot active data and cold unchanging data – it is called the warm-tier. It is warm because the data within it is still relatively active and may even be occasionally updated. It is part of the HANA database, and is maintained with transactional consistency. It does not have the extreme performance of HANA memory, but still offers very good performance for analytical type queries.
Data is kept outside of HANA memory, but in an integrated disk-backed tier – this tier is termed as Dynamic Tier (DT) - it should not used to store hot data. If you have a lot of hot data, you should ensure that you have a large enough HANA system to process it. The purpose of dynamic tiering is not to create a cheap HANA system, but to offload less frequently accessed data to a tier with lower price-performance ratio.
The tiers are colored with red for hot, orange for warm, and shades of blue for cold. There is a dotted line around the hot and warm tiers of the HANA database, indicating that the tiers are tightly integrated and treated as two stores of a single, unified database. The hot tier of a HANA system is HANA memory. While HANA is composed of only a hot tier, HANA used various memory displacement and scale-out approaches to manage less frequently accessed data - choices for the cold tier include SAP IQ, external databases, or Hadoop.
In a HANA system, there are currently two available technologies for a warm data tier, tightly integrated with the HANA database. Those two choices are HANA Dynamic Tiering (DT) and HANA extension nodes.
A HANA extension node is a normal HANA node with relaxed memory requirements. Instead of only half of HANA memory being available for user data, twice the memory is available for user data – this is because the data is stored in the HANA persistence layer and only brought into memory when needed. Therefore, you want to keep only infrequently accessed data in a HANA extension node. Otherwise, you will be paging data in and out all the time and thrashing the system. Extension nodes are used by SAP BW and are also available for HANA data marts. A HANA extension node is a pure HANA node and includes all the specialized HANA engines, such as graph, spatial, and Predictive Analytics.
Data Tiering (DT) is the integration of a disk-backed column store based on SAP IQ technology with the HANA database. Dynamic tiering is installed on its own server, but is not a pure HANA node. You can execute SQL and calculation views on HANA dynamic tiering data. Query processing is pushed down to dynamic tiering and data does not have to be brought into HANA memory to run SQL commands on it.
However, if you want to run more exotic HANA functions, such as predictive or text analytics on dynamic tiering data, that data will need to be brought into HANA memory first before it can be processed.
Dynamic tiering can be used in HANA-native data marts and a few SAP applications, like SAP CAR (Customer Activity Repository). These applications use HANA dynamic tiering to store some of the older, less frequently accessed data. SAP BW does not use dynamic tiering. But the S/4HANA team is evaluating using dynamic tiering as an option to paged attributes when there are very large volumes of data in S/4HANA.
When you install dynamic tiering on your HANA system, you will have two new types of HANA tables to use - an extended table and a multistore table. Both of these tables show up in the HANA catalog. An extended table is a column table with all of its data in the dynamic tiering extended store. A multistore table is a partition table with some partitions in HANA memory and some partitions in the extended store.
You can execute SQL queries on these types of tables. And extended and multistore tables can be used in calculation views. HANA dynamic tiering is tightly integrated with the HANA database. The dynamic tiering server is part of the HANA landscape and can be added to either a HANA scale-up or scale-out system.
Dynamic tiering is installed with the HANA installer and uses the same administration and monitoring tools – HANA studio and HANA cockpit. All interaction to dynamic tiering occurs through a HANA connection.
When you back up the HANA database, both stores are backed up together. System replication replicates both stores to a secondary system.
The HANA query optimizer breaks queries up into fragments and pushes the fragments that interact with dynamic tiering data to the dynamic tiering server. Query processing occurs in dynamic tiering, which returns the reduced result set back to HANA.
Dynamic tiering is designed to be used for analytical queries against largely read-only data. OLTP-style row-based transactions, such as single updates, deletes, inserts, should be executed against HANA data in memory, and then occasionally partitions of data moved in bulk from memory to the extended store for best performance. The state of data management in HANA dynamic tiering is an integrated tier with HANA.
Only a single dynamic tiering server is allowed for each HANA tenant database. A dynamic tiering server can manage about 100 terabytes of compressed data on disk, that is, a single server can manage a lot of data and a lot of concurrent users hitting the extended store.To reach even greater volumes of warm data, a scale-out dynamic tiering server configuration is being considered.
In the area of backup and recovery, Dynamic tiering supports both file-based backups and backups to third-party backup tool providers using the back-end interface. Dynamic tiering supports full and delta backups and also log backups for point-in-time recovery. One backup approach that dynamic tiering does not support is storage snapshots.
For high availability and disaster recovery, you can set up a failover server for dynamic tiering. Dynamic tiering also supports two-tier and three-tier system replication. The support for cluster managers is partially available, but requires some completion items.