[
https://issues.apache.org/jira/browse/HBASE-16981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668595#comment-15668595
]
huaxiang sun edited comment on HBASE-16981 at 11/15/16 10:43 PM:
-----------------------------------------------------------------
One question, today the mob compact chore can be controlled by how often it
needs to run with
MOB_COMPACTION_CHORE_PERIOD (the default is one week)
And with MOB_COMPACTION_MERGEABLE_THRESHOLD, it can be configured that files
larger than the threshold will be skipped by the minor mob compact.
Are these not enough to reduce IO? assuming that the mob compact chore causes
the situation Anoop described, not the manual mob compaction.
was (Author: huaxiang):
One question, today the mob compact chore can be controlled by how often it
needs to run with
MOB_COMPACTION_CHORE_PERIOD (the default is one week)
And with MOB_COMPACTION_MERGEABLE_THRESHOLD, it can be configured that files
larger than the threshold will be compacted by the minor mob compact.
Are these not enough to reduce IO? assuming that the mob compact chore causes
the situation Anoop described, not the manual mob compaction.
> Expand Mob Compaction Partition policy from daily to weekly, monthly and
> beyond
> -------------------------------------------------------------------------------
>
> Key: HBASE-16981
> URL: https://issues.apache.org/jira/browse/HBASE-16981
> Project: HBase
> Issue Type: New Feature
> Components: mob
> Affects Versions: 2.0.0
> Reporter: huaxiang sun
> Assignee: huaxiang sun
> Attachments: HBASE-16981.master.001.patch,
> HBASE-16981.master.002.patch,
> Supportingweeklyandmonthlymobcompactionpartitionpolicyinhbase.pdf
>
>
> Today the mob region holds all mob files for all regions. With daily
> partition mob compaction policy, after major mob compaction, there is still
> one file per region daily. Given there is 365 days in one year, at least 365
> files per region. Since HDFS has limitation for number of files under one
> folder, this is not going to scale if there are lots of regions. To reduce
> mob file number, we want to introduce other partition policies such as
> weekly, monthly to compact mob files within one week or month into one file.
> This jira is create to track this effort.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
