[
https://issues.apache.org/jira/browse/HBASE-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Biju Nair updated HBASE-21000:
------------------------------
Component/s: Compaction
> Default limits for PressureAwareCompactionThroughputController are too low
> --------------------------------------------------------------------------
>
> Key: HBASE-21000
> URL: https://issues.apache.org/jira/browse/HBASE-21000
> Project: HBase
> Issue Type: Improvement
> Components: Compaction
> Affects Versions: 1.5.0
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: HBASE-21000-branch-1.patch, HBASE-21000.patch
>
>
> In PressureAwareCompactionThroughputController:
> {code:java}
> /**
>
>
> * A throughput controller which uses the follow schema to limit throughput
>
>
> * <ul>
>
>
> * <li>If compaction pressure is greater than 1.0, no limitation.</li>
>
>
> * <li>In off peak hours, use a fixed throughput limitation
>
>
> * {@value #HBASE_HSTORE_COMPACTION_MAX_THROUGHPUT_OFFPEAK}</li>
>
>
> * <li>In normal hours, the max throughput is tuned between
>
>
> * {@value #HBASE_HSTORE_COMPACTION_MAX_THROUGHPUT_LOWER_BOUND} and
>
>
> * {@value #HBASE_HSTORE_COMPACTION_MAX_THROUGHPUT_HIGHER_BOUND}, using the
> formula "lower +
>
> * (higher - lower) * compactionPressure", where compactionPressure is
> in range [0.0, 1.0]</li>
>
> * </ul>
>
>
> */
> {code}
> The lower and upper bounds are 10MB/sec and 20MB/sec, respectively:
> {code:java}
> public static final String
> HBASE_HSTORE_COMPACTION_MAX_THROUGHPUT_HIGHER_BOUND =
> "hbase.hstore.compaction.throughput.higher.bound";
> private static final long
> DEFAULT_HBASE_HSTORE_COMPACTION_MAX_THROUGHPUT_HIGHER_BOUND =
> 20L * 1024 * 1024;
> public static final String
> HBASE_HSTORE_COMPACTION_MAX_THROUGHPUT_LOWER_BOUND =
> "hbase.hstore.compaction.throughput.lower.bound";
> private static final long
> DEFAULT_HBASE_HSTORE_COMPACTION_MAX_THROUGHPUT_LOWER_BOUND =
> 10L * 1024 * 1024;
> {code}
> (In contrast, in PressureAwareFlushThroughputController the lower and upper
> bounds are 10x of those limits, at 100MB/sec and 200MB/sec, respectively.)
> In fairly light load scenarios we see compaction quickly falls behind and
> write clients are backed off or failing due to RegionTooBusy exceptions.
> Although compaction throughput becomes unbounded after the store reaches the
> blocking file count, in the lead up to this the default settings do not
> provide enough bandwidth to stave off blocking. The defaults should be
> increased.
> I'm not sure what new good defaults make sense. We could start by doubling
> them to 20MB/sec and 40MB/sec respectively. Might need to be doubled again.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
