[
https://issues.apache.org/jira/browse/HBASE-30405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wenhao reassigned HBASE-30405:
------------------------------
Assignee: (was: wenhao)
> [Balancer] `HDFSBlocksDistribution.getTopHosts()` breaks weight ties by
> hostname, causing the current RS to be misranked and occasional zero-gain
> region moves
> --------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-30405
> URL: https://issues.apache.org/jira/browse/HBASE-30405
> Project: HBase
> Issue Type: Bug
> Components: Balancer
> Affects Versions: 2.0.0, 2.5.11
> Reporter: wenhao
> Priority: Trivial
>
> h2. Summary
> When multiple hosts hold an equal total weight of a region's HFiles (i.e.
> equal block locality), {{HDFSBlocksDistribution.getTopHosts()}} resolves the
> tie by hostname. The resulting "most local host" ordering is therefore
> arbitrary with respect to locality, and the host currently serving the region
> is frequently not ranked first even though its locality is identical to the
> top host. This contradicts the documented contract of
> {{BalancerClusterState.regionLocations}} ("regionIndex -> list of serverIndex
> sorted by locality") and, through the most-local-entity selection used by
> {{{}LocalityBasedCandidateGenerator{}}}, can make the stochastic balancer
> emit zero-locality-gain move/swap proposals.
> h2. Problem
> {{HDFSBlocksDistribution.getTopHostsWithWeights()}} builds a {{TreeSet}}
> ordered by {{HostAndWeight.WeightComparator}} and returns its
> {{{}descendingSet(){}}}:
> * {{{}WeightComparator.compare(l, r){}}}: primary key is weight (ascending);
> *when weights are equal, the tie is broken by
> {{l.getHost().compareTo(r.getHost())}}* (hostname, ascending).
> * {{descendingSet()}} reverses that order, so the returned list is: weight
> descending, and {*}among equal weights, hostname descending{*}.
> Consequently the first element of {{getTopHosts()}} is the host with the
> maximum block weight; if several hosts share the same maximum weight, the
> first one is simply the lexicographically largest hostname — not the host
> with any locality advantage.
> h2. Root cause
> # {{HDFSBlocksDistribution.java}}
> ** {{WeightComparator}} tie-break: {{return
> l.getHost().compareTo(r.getHost());}}
> ** {{getTopHostsWithWeights()}} returns
> {{{}orderedHosts.descendingSet(){}}}, so equal-weight hosts are ordered by
> hostname {_}descending{_}.
> # {{RegionLocationFinder.getTopBlockLocations(RegionInfo region)}} returns
> the host list in exactly that order.
> # {{BalancerClusterState.registerRegion()}} fills
> {{regionLocations[regionIndex]}} from that list. The field contract says the
> list is "sorted by locality", but under weight ties it is actually sorted by
> hostname.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
