[
https://issues.apache.org/jira/browse/HBASE-24289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17118273#comment-17118273
]
Guanghao Zhang commented on HBASE-24289:
----------------------------------------
{quote}any other date-based compaction policies or other writes (wals and cf
maybe?)
{quote}
All wal is created for current write. It doesn't need date-based.
{quote}A writer factory that returns writers on different storage/folder based
on date ranges would be a more reusable component. We can call it date-based
HDFS storage assignment.
{quote}
I thought this shoule be a new issue. It is not this proposal's goal. And now
the 2.x hdfs didn't have api about create a file writer with storage policy.
The api only finish write first then set storage policy. Then move the data to
meet storage policy. So this is not friendly.
{quote}Another change I recommend is for the policy to take a list of date
boundaries for ranges to assign (potentially) different HDFS storages. It can
be empty so all are on the same storage, or multiple for different storages.
This makes it backward compatible and easy for future extension or adoption.
{quote}
Can you explain more about this? The hot, warm, cold classification is not
enough?
> Heterogeneous Storage for Date Tiered Compaction
> ------------------------------------------------
>
> Key: HBASE-24289
> URL: https://issues.apache.org/jira/browse/HBASE-24289
> Project: HBase
> Issue Type: New Feature
> Components: Compaction
> Reporter: Mengqing Peng
> Assignee: Mengqing Peng
> Priority: Major
>
> Support DateTiredCompaction(HBASE-15181) for cold and hot data separation,
> support different storage policies for different time periods of data to get
> better performance, for example, we can configure the data of last 1 month in
> SSD, and 1 month ago data was in HDD.
> design doc:
> https://docs.google.com/document/d/1fk_EWLNnxniwt3gDjUS_apQ3cPzn90AmvDT1wkirvKE/edit?usp=sharing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
