[
https://issues.apache.org/jira/browse/HBASE-27496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17635993#comment-17635993
]
Charles Connell commented on HBASE-27496:
-----------------------------------------
Because the split planning and merge planning works quite differently, it's
simpler to implement the spirit of this ticket as two independent counters. In
order to make clear that these are two independent counters, I made two
settings. The limits could also be controlled by a single setting, but I still
think the limit should be applied twice, separately. This gives splits and
merges both a fair chance at making it into the plans.
> Limit size of plans produced by SimpleRegionNormalizer
> ------------------------------------------------------
>
> Key: HBASE-27496
> URL: https://issues.apache.org/jira/browse/HBASE-27496
> Project: HBase
> Issue Type: Improvement
> Components: Normalizer
> Reporter: Charles Connell
> Priority: Minor
>
> My company (Hubspot) is starting to use {{{}SimpleRegionNormalizer{}}}. We
> turn the normalizer switch on for 30 minutes each day, when our database
> traffic is at a low point. We're using the
> {{hbase.normalizer.throughput.max_bytes_per_sec}} setting to create a rate
> limit. I've found that while the {{SimpleRegionNormalizer}} only produces new
> plans for 30 minutes each day, the plans often take many hours to execute.
> This leds to region splits, merges, and moves occurring in our HBase clusters
> during hours we'd prefer them not to.
> I propose two new settings:
> * {{hbase.normalizer.merge.plans_size_limit.mb}}
> * {{hbase.normalizer.split.plans_size_limit.mb}}
> This will allow HBase administrators to limit the number of plans produced by
> a run of {{{}SimpleRegionNormalizer{}}}, by forcing it to stop producing new
> plans once the cumulative region size limits are exceeded. This will give you
> a way to limit approximately how long it takes to execute the plans. Because
> the current limit to execute plans is primarily determined by a per-byte rate
> limit, I propose that the new settings also work on a similar basis. This
> will make it feasible to reason about how your rate limit and your size
> limits interact.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
