[
https://issues.apache.org/jira/browse/HBASE-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-8329:
-------------------------
Release Note:
Adds compaction throughput limit mechanism(the word "throttle" is already used
when choosing compaction thread pool, so use a different word here to avoid
ambiguity). Default is
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController,
will limit throughput as follow:
1. In off peak hours, use a fixed limitation
"hbase.hstore.compaction.throughput.offpeak" (default is Long.MAX_VALUE which
means no limitation).
2. In normal hours, the limitation is tuned between
"hbase.hstore.compaction.throughput.lower.bound"(default 10MB/sec) and
"hbase.hstore.compaction.throughput.higher.bound"(default 20MB/sec), using the
formula "lower + (higer - lower) * param" where param is in range [0.0, 1.0]
and calculate based on store files count on this regionserver.
3. If some stores have too many store files(storefilesCount >
blockingFileCount), then there is no limitation no matter peak or off peak.
You can set "hbase.regionserver.throughput.controller" to
org.apache.hadoop.hbase.regionserver.throttle.NoLimitThroughputController to
disable throughput controlling.
And we have implemented ConfigurationObserver which means you can change all
configurations above and do not need to restart cluster.
The throttle is on by default in hbase-2.0.0. There is no limit in hbase-1.x.
was:
Adds compaction throughput limit mechanism(the word "throttle" is already used
when choosing compaction thread pool, so use a different word here to avoid
ambiguity). Default is
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController,
will limit throughput as follow:
1. In off peak hours, use a fixed limitation
"hbase.hstore.compaction.throughput.offpeak" (default is Long.MAX_VALUE which
means no limitation).
2. In normal hours, the limitation is tuned between
"hbase.hstore.compaction.throughput.lower.bound"(default 10MB/sec) and
"hbase.hstore.compaction.throughput.higher.bound"(default 20MB/sec), using the
formula "lower + (higer - lower) * param" where param is in range [0.0, 1.0]
and calculate based on store files count on this regionserver.
3. If some stores have too many store files(storefilesCount >
blockingFileCount), then there is no limitation no matter peak or off peak.
You can set "hbase.regionserver.throughput.controller" to
org.apache.hadoop.hbase.regionserver.throttle.NoLimitThroughputController to
disable throughput controlling.
And we have implemented ConfigurationObserver which means you can change all
configurations above and do not need to restart cluster.
> Limit compaction speed
> ----------------------
>
> Key: HBASE-8329
> URL: https://issues.apache.org/jira/browse/HBASE-8329
> Project: HBase
> Issue Type: Improvement
> Components: Compaction
> Reporter: binlijin
> Assignee: Duo Zhang
> Fix For: 2.0.0, 1.1.0, 0.98.11
>
> Attachments: HBASE-8329-0.98-addendum.patch, HBASE-8329-0.98.patch,
> HBASE-8329-10.patch, HBASE-8329-11.patch, HBASE-8329-12.patch,
> HBASE-8329-2-trunk.patch, HBASE-8329-3-trunk.patch, HBASE-8329-4-trunk.patch,
> HBASE-8329-5-trunk.patch, HBASE-8329-6-trunk.patch, HBASE-8329-7-trunk.patch,
> HBASE-8329-8-trunk.patch, HBASE-8329-9-trunk.patch,
> HBASE-8329-branch-1.patch, HBASE-8329-trunk.patch, HBASE-8329_13.patch,
> HBASE-8329_14.patch, HBASE-8329_15.patch, HBASE-8329_16.patch,
> HBASE-8329_17.patch
>
>
> There is no speed or resource limit for compaction，I think we should add this
> feature especially when request burst.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
