[
https://issues.apache.org/jira/browse/HBASE-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-8329:
-------------------------
Release Note:
Adds compaction throughput limit mechanism(the word "throttle" is already used
when choosing compaction thread pool, so use a different word here to avoid
ambiguity). Default is
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController,
will limit throughput as follow:
1. In off peak hours, use a fixed limitation
"hbase.hstore.compaction.throughput.offpeak" (default is Long.MAX_VALUE which
means no limitation).
2. In normal hours, the limitation is tuned between
"hbase.hstore.compaction.throughput.lower.bound"(default 10MB/sec) and
"hbase.hstore.compaction.throughput.higher.bound"(default 20MB/sec), using the
formula "lower + (higer - lower) * param" where param is in range [0.0, 1.0]
and calculate based on store files count on this regionserver.
3. If some stores have too many store files(storefilesCount >
blockingFileCount), then there is no limitation no matter peak or off peak.
You can set "hbase.regionserver.throughput.controller" to
org.apache.hadoop.hbase.regionserver.throttle.NoLimitThroughputController to
disable throughput controlling.
And we have implemented ConfigurationObserver which means you can change all
configurations above and do not need to restart cluster.
was:
Adds compaction throughput limit mechanism(the word "throttle" is already used
when choosing compaction thread pool, so use a different word here to avoid
ambiguity). Default is
org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController,
will limit throughput as follow:
1. In off peak hours, use a fixed limitation
"hbase.hstore.compaction.throughput.offpeak" (default is Long.MAX_VALUE which
means no limitation).
2. In normal hours, the limitation is tuned between
"hbase.hstore.compaction.throughput.lower.bound"(default 10MB/sec) and
"hbase.hstore.compaction.throughput.higher.bound"(default 20MB/sec), using the
formula "lower + (higer - lower) * param" where param is in range [0.0, 1.0]
and calculate based on store files count on this regionserver.
3. If some stores have too many store files(storefilesCount >
blockingFileCount), then there is no limitation no matter peak or off peak.
You can set "hbase.regionserver.throughput.controller" to
org.apache.hadoop.hbase.regionserver.compactions.NoLimitCompactionThroughputController
to disable throughput controlling.
And we have implemented ConfigurationObserver which means you can change all
configurations above and do not need to restart cluster.
> Limit compaction speed
> ----------------------
>
> Key: HBASE-8329
> URL: https://issues.apache.org/jira/browse/HBASE-8329
> Project: HBase
> Issue Type: Improvement
> Components: Compaction
> Reporter: binlijin
> Assignee: Duo Zhang
> Fix For: 2.0.0, 1.1.0, 0.98.11
>
> Attachments: HBASE-8329-0.98-addendum.patch, HBASE-8329-0.98.patch,
> HBASE-8329-10.patch, HBASE-8329-11.patch, HBASE-8329-12.patch,
> HBASE-8329-2-trunk.patch, HBASE-8329-3-trunk.patch, HBASE-8329-4-trunk.patch,
> HBASE-8329-5-trunk.patch, HBASE-8329-6-trunk.patch, HBASE-8329-7-trunk.patch,
> HBASE-8329-8-trunk.patch, HBASE-8329-9-trunk.patch,
> HBASE-8329-branch-1.patch, HBASE-8329-trunk.patch, HBASE-8329_13.patch,
> HBASE-8329_14.patch, HBASE-8329_15.patch, HBASE-8329_16.patch,
> HBASE-8329_17.patch
>
>
> There is no speed or resource limit for compaction，I think we should add this
> feature especially when request burst.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
