[
https://issues.apache.org/jira/browse/HBASE-21014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16576290#comment-16576290
]
Hari Sekhon edited comment on HBASE-21014 at 8/10/18 1:33 PM:
--------------------------------------------------------------
I thought that was really the crux of it:
Write the HDFS location preference hints the same as the FavoredNodeBalancer
does while applying all the usual Stochastic Balancer balancing heuristics to
make sure regions and load are evenly spread. Since the HBase Balancer chooses
where to move regions to it can update the block location preferences metadata
to match it whenever it migrates regions.
That way if I need to rebalance HDFS blocks, the HDFS Balancer won't move the
region blocks out of their primary active region locations when hdfs block
pinning is enabled.
was (Author: harisekhon):
I thought this was really the crux of it:
Write the HDFS location preference hints the same as the FavoredNodeBalancer
does while applying all the usual Stochastic Balancer balancing heuristics to
make sure regions and load are evenly spread.
That way if I need to rebalance HDFS blocks, the HDFS Balancer won't move the
region blocks out of their primary active region locations when hdfs block
pinning is enabled.
> Improve Stochastic Balancer to write HDFS favoured node hints for region
> primary blocks to avoid destroying data locality if needing to use HDFS
> Balancer
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-21014
> URL: https://issues.apache.org/jira/browse/HBASE-21014
> Project: HBase
> Issue Type: Improvement
> Components: Balancer
> Affects Versions: 1.1.2
> Reporter: Hari Sekhon
> Priority: Major
>
> Improve Stochastic Balancer to include the HDFS region location hints to
> avoid HDFS Balancer destroying data locality.
> Right now according to a mix of docs, jiras and mailing list info it appears
> that one must change
> {code:java}
> hbase.master.loadbalancer.class{code}
> to the org.apache.hadoop.hbase.favored.FavoredNodeLoadBalancer as it looks
> like this functionality is only within FavoredNodeBalancer and not the
> standard Stochastic Balancer.
> [http://hbase.apache.org/book.html#_hbase_and_hdfs]
> This is not ideal because we'd still like to use all the heuristics and work
> that has gone in the Stochastic Balancer which I believe right now is the
> best and most mature HBase balancer.
> See also the linked Jiras and this discussion:
> [http://apache-hbase.679495.n3.nabble.com/HDFS-Balancer-td4086607.html]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
