[
https://issues.apache.org/jira/browse/HBASE-24289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17117957#comment-17117957
]
Clara Xiong commented on HBASE-24289:
-------------------------------------
This is a great idea to leverage multiple types of storage for data of
different age. May I suggest a more generic design? In my opinion, this is
orthogonal to date-tied compaction. It can be overlaid on top of DTC, any other
date-based compaction policies or other writes (wals and cf maybe?). A writer
factory that returns writers on different storage/folder based on date ranges
would be a more reusable component. We can call it date-based HDFS storage
assignment. And I would call it date ranges to differentiate from windows for
date-tiered compaction. Another change I recommend is for the policy to take a
list of date boundaries for ranges to assign (potentially) different HDFS
storages. It can be empty so all are on the same storage, or multiple for
different storages. This makes it backward compatible and easy for future
extension or adoption.
> Heterogeneous Storage for Date Tiered Compaction
> ------------------------------------------------
>
> Key: HBASE-24289
> URL: https://issues.apache.org/jira/browse/HBASE-24289
> Project: HBase
> Issue Type: New Feature
> Components: Compaction
> Reporter: Mengqing Peng
> Assignee: Mengqing Peng
> Priority: Major
>
> Support DateTiredCompaction(HBASE-15181) for cold and hot data separation,
> support different storage policies for different time periods of data to get
> better performance, for example, we can configure the data of last 1 month in
> SSD, and 1 month ago data was in HDD.
> design doc:
> https://docs.google.com/document/d/1fk_EWLNnxniwt3gDjUS_apQ3cPzn90AmvDT1wkirvKE/edit?usp=sharing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
