[
https://issues.apache.org/jira/browse/HBASE-28513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ray Mattingly updated HBASE-28513:
----------------------------------
Release Note:
This adds support for discrete rules to be configured for the balancer. For
example, you can configure the following conditional rules:
1. Set `hbase.master.balancer.stochastic.conditionals.distributeReplicas` to
true to use a newer, more strict replica balancing algorithm. This improves
upon cost-based replica balancing in that it does not squash other load
balancing considerations, it separates secondary replicas from one another, it
will trigger balancer runs based on rack colocation, and it will seek
perfection rather than naive appeasement of the balancer's cost targets.
2. Set `hbase.master.balancer.stochastic.conditionals.isolateMetaTable` to true
to isolate meta table regions onto their own RegionServer. This can improve
reliability on large clusters that may be prone to meta hotspotting, with only
a minor cost inefficiency.
3. Set `hbase.master.balancer.stochastic.conditionals.isolateSystemTables` to
true to isolate system tables to their own RegionServer. Unlike meta isolation,
this will group all system tables together on one RegionServer (or _n_
RegionServers, with secondary replicas in the mix). This works in tandem with
meta table isolation; with both enabled, meta regions will live on their own
server, and system tables will live on another.
If perfect rule compliance cannot be achieved — for example, due to an
insufficient number of living RegionServers — then the balancer will simply
keep trying periodically, and allow your regions to be assigned in violation of
the desired rules.
> The StochasticLoadBalancer should support discrete evaluations for replica
> distribution
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-28513
> URL: https://issues.apache.org/jira/browse/HBASE-28513
> Project: HBase
> Issue Type: Improvement
> Components: Balancer
> Reporter: Ray Mattingly
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2
>
>
> I have a larger write up available
> [here|https://docs.google.com/document/d/1jA8Ghs86v7b-53j5DcsdbPnOXxbHjewkIBFi1E4S1pY/edit?usp=sharing].
> Secondary replica balancing squashes all other cost considerations.
> Basically there are a few cost functions with relatively huge default
> multipliers. For example `PrimaryRegionCountSkewCostFunction` has a default
> multiplier of 100,000. Meanwhile things like StoreFileCostFunction have a
> multiplier of 5. Having any multiplier of 100k, while others are single
> digit, basically makes the latter category totally irrelevant from balancer
> considerations.
> I understand that it's critical to distribute a region's replicas across
> multiple hosts/racks, but I don't think we should do this at the expense of
> all other balancer considerations.
> For example, maybe we could have two types of balancer considerations: costs
> (as we do now), and conditionals (for the more discrete considerations, like
> ">1 replica of the same region should not exist on a single host"). This
> would allow us to prioritize replica distribution _and_ maintain
> consideration for things like storefile balance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
