Auto Tiering (QTiering 2.0)
 
QSAN Auto Tiering (QTiering) is an automated storage tiering solution that automatically places hot data on SSD or faster hard drives (performance tier) and cold data on lower cost high-capacity drives (capacity tier), allowing you to optimize application performance without straining your budget or sacrificing capacity. Auto tiering is a licensed feature available on the XCubeSAN series.
 
*
Storage Efficiency on Demand
 
From the perspective of storage features, the performance of SSD is high, but the cost is also high per GB. Relatively speaking, the cost of traditional hard drive is low, so as performance is relatively poor. If we follow the 80/20 rule to configure storage systems, all-SSD configuration is unreasonable for all but the most intensive applications. On the other hand, the data itself has a lifecycle. As the age of the data increases, it is accessed less often.

Therefore, to balance performance and cost factors, adapting a hybrid storage architecture with a mixture of SSD and traditional HDD seems to be the most reasonable approach for modern IT environments. An automated tiering pool is a simple and elegant solution for dynamically matching storage requirements with changes in the frequency of data access.
*
3 Levels of Tiered Storage
 
QTiering supports 3 tiers of different drive types.
 
  • Tier 1: SAS/SATA* 2.5” SSD drive
  • Tier 2: 15K/10K 3.5”/2.5” SAS drive
  • Tier 3: 7.2K near-line 3.5”/2.5” SAS drive
*
Auto Tiering Architecture
 
A newly created auto tiering pool is based on thin provisioning technology. Each tier works based on one or more disk groups. To increase the capacity of an auto tiering pool, any tier (disk group) which contains either one tier of SSDs, SAS HDDs, or NL-SAS HDDs can be added to the pool any time. The following is the storage architecture of an auto tiering pool.
 
*
*
Flexible RAID & Disk Configurations
 
Auto Tiering 2.0 supports flexible RAID and disk configurations. You can create each tier (disk group) with different RAID level and different quantity of disk. For example, SSD tier uses 4 disks with RAID 10 for extreme performance, SAS tier uses 6 disks with RAID 6, and NL-SAS tier uses 8 disks with RAID 5 for capacity. This feature is very important for IT administrator to arrange storage plan flexibly.
Intelligent QTiering Mechanism
 
QTiering, an auto tiering storage management system, manages the data relocation and monitors the data hotness ratio using half-life coefficient and advanced ranking algorithm. QTiering operates on three major functions.
 
  • Sub-LUN Statistics
    The volume is divided into 1GB units, which is called a sub-LUN. This is the basic unit of data movement among tiers. Whenever there are I/O requests, the activity level of a sub LUN is determined by counting the read and write frequency to the sub-LUN.

  • Ranking Algorithm
    Access records of each sub-LUN are collected and analyzed every hour. LVM maintains a cumulative I/O count and weighs each I/O by how recently it arrived, using a half-life coefficient. The ranking algorithm then uses these statistics to calculate the percentage of hot data.

  • Data Movement
    The data relocation engine then uses these percentages as guidance to move sub-LUNs between storage tiers automatically. The data relocation process will not stop I/O services. When data relocation begins to move sub-LUNs from slower tier to the faster tier, you will notice the I/O performance increases over time.
*
Well Defined Tiering Policies
 
For a best performance in various environments, auto tiering has a completely automated feature that implements a set of tiering polices. Tiering policies determine how new allocations and ongoing relocations should apply within a volume for those requirements.
 
  • Auto Tiering (Default)
    Sets the initial data placement to the optimized tier (disk group) and then relocates the data based on the statistics such that data is relocated among tiers according to the I/O activity.

  • Start Highest then Auto Tiering
    First sets the preferred tier for the initial data placement to the highest tiers with available space, then relocates the data based on the statistics and the auto tiering algorithm.

  • Highest Available Tier
    Sets the preferred tier for the initial data placement to the highest tiers with available space, and so as the succeeding data relocation.

  • Lowest Tier
    Sets the preferred tier for the initial data placement to the lowest tiers with available space, and so as the succeeding data relocation.

  • No Data Movement
    Sets the preferred tier for the initial data to the optimized tier, and retain the data without movement.
*
Benefits of Using QTiering Technology
 
  • Flexible Configurations
    IT administrator can arrange the storage plan flexibly by QTiering flexible architecture. And also easy to setup by the intuitive SANOS 4.0 web UI.

  • Optimal Storage Efficiency
    According to the data access frequency, continue optimizing the data across extreme performance SSD (SSD tier), performance SAS HDDs (SAS tier), and high-capacity NL-SAS HDDs (NL-SAS tier).

  • Cost Saving
    Purchasing only the required SSDs can lower the TCO (Total Cost of Ownership).

  • High performance
    Intelligently managing data placement can increase performance by moving the hot data to the performance tier.

  • Schedule Relocation
    The relocation of auto tiering can be scheduled during off-peak hours to avoid affecting the system performance in office hours.

  • Tiering Analysis
    By monitoring the tiering status on each pool, you can dynamically adjust to more accurate policy.
*
Ideal for Data Center Environments
 
SANOS with QTiering technology provides a solution to achieve optimal storage efficiency and improved performance, making it the most cost effective storage solution for data center environments with dynamic workload changes.
*
SSD Cache vs. Auto Tiering
 
The SSD cache and auto tiering solutions can work together, and complement each other. A key difference between tiering and cache is that tiering moves data to SSD instead of simply caching it. Tiering can also move data both from slower storage to faster storage and vice versa. But SSD cache is essentially a one-way transaction. The important difference between moves and copies is that a cache does not need to have the redundancy that tiering does.
 
Total storage capacity in auto tiering is a sum of all individual tier capacities whereas in cache, the cache capacity does not add to the overall slower storage capacity. This is one of the key differences. In addition, SSD cache affects more rapidly than auto tiering because auto tiering will take effect after data relocation. So SSD cache warm-up timeframe is usually minutes/hours whereas tiering warm-up is usually in days.
 
SSD cache is used for highly frequent data access environment and is effective in a short term such as virtualization or video editing applications. However, auto tiering is used for predictable I/O workloads and is effective in a long term. It's suitable for web, file, or email server applications.
 
 SSD CacheAuto Tiering
Total CapacityHDDHDD + SSD
When SSD is DamagedPool Works FinePool Fails
PerformanceEffective in Short TermEffective in Long Term
*
Best Practice
Configuration Planning Advice
 
The following is a general guide to the auto tiering pool planning. The user can fine-tune according to the actual situation.
 
  • SSD Tier ($$$)
    Suggest SSD tier using at least 4 disks with RAID 10 (better) or 2 disks with RAID 1 for extreme performance. Prepare SSD storage capacity in 10% to 15% of the total pool capacity to fulfill the requirements of critical high I/O applications.

  • SAS Tier ($$)
    Suggest SAS HDD tier configuring with RAID 6 (better) or RAID 5. Prepare about 30% of the total storage capacity.

  • NL-SAS Tier ($)
    For capacity tier, suggest NL-SAS HDD using RAID 5 level to store cold data. This tier occupies the rest of the storage capacity.
 
Suggest using "Auto Tiering" policy when creating a volume, and the relocation schedule remains in daily. These configurations show the best performance for auto tiering.
*
Take an example for reference. First, you can estimate the total capacity used, and estimate how much hot data or high I/O your application uses every day. Assuming 666GB per day, the recommended SSD tier capacity is at least 1.5 times, 1.5 x 666GB = 1TB, as a conservative estimate. And then calculate the SAS HDD tier capacity about 3 times of the SSD tier capacity, 3 x 1TB = 3TB, as if the SSD tier full of the buffer, so that the performance does not drop too much. This tier is optional. The remaining space left for NL-SAS HDD tier. The following table is the summary for reference.
 
TierCapacity per DriveQuantityRAID LevelCapacity per TierCapacity Ratio
SAS SSD Tier500GB4RAID 10(4/2) x 500GB = 1TB10%
SAS HDD Tier1TB5RAID 6(5-2) x 1TB = 3TB30%
NL-SAS HDD Tier3TB3RAID 5(3-1) x 3TB = 6TB60%
 
This is a rough planning proposal. Whether to meet customer requirements also requires user to calculate the performance and necessary capacity. Of course, if more capacity is needed, you can also add a disk group to any tier.
Cases 1: Video Editing
 
We assume that video editing has the characteristics of focus data over a period of time. When user edits a new video, the video remains at the SSD tier and performs extreme performance. After the editing is complete, the video moves to the HDD tier and leaves the space for the next video. Therefore, we recommend setting the auto-tiering policy to Start Highest then Auto Tiering.
 
For more details, please refer to the Auto Tiering 2.0 White Paper.
*
Cases 2: VMware
 
We simulate 8 VMs (Virtual Machines) running on a server, assume that they have different I/O queue depths and possess intensive I/O flows. We recommend setting the auto-tiering policy as Auto Tiering.
 
At the beginning, 8 VMs run I/O by IOmeter. After working a while, Stop VM2~VM8 I/O but keep VM1 running I/O. The data in VM1 will be accessed more frequently than others. After analysis and relocation by auto tiering mechanism, the data in VM1 has been moved to higher tier. At last, run VM2~VM8 I/O again, record the performance.
 
The following table summarizes the throughput before and after the relocation. And calculate the percentage of improvement as a reference. This verifies the scenario and meets the expectations of VMware.
 
VM NameThroughput Before RelocationThroughput After RelocationImproved
VM19.96 MB/s465.86 MB/s4,577%
VM24.78 MB/s74.75 MB/s1,464%
VM34.41 MB/s68.78 MB/s1,460%
VM44.13 MB/s63.59 MB/s1,440%
VM53.98 MB/s60.03 MB/s1,408%
VM63.79 MB/s57.12 MB/s1,407%
VM73.70 MB/s54.90 MB/s1,384%
VM83.61 MB/s54.18 MB/s1,401%
 
For more details, please refer to the Auto Tiering 2.0 White Paper.
*
Cases 3: Reaction Suddenly
 
In order to cope with an expected sudden event, IT administrator can move the required data to the SSD tier in advance. In general, we recommend setting the auto-tiering policy to Lowest Tier. The day before the activity, IT administrator manually set the volume containing the required data to Highest Available Tier and then performs Relocation Now manually to force relocating data.
 
For more details, please refer to the Auto Tiering 2.0 White Paper.
Conclusion
 
With auto tiering technology, the XCubeSAN series can help you put the right data at the right place in the right time for optimal use of all storage tiers and allow you to reduce storage costs and management overhead while increasing performance and capacity.
 
Intelligent algorithm behind auto tiering manages the data relocation and monitors the data hotness ratio using half-life coefficient and advanced ranking mathematics. Relocations can occur on the user-defined relocation schedule, making auto tiering a truly automated offering.