[
https://issues.apache.org/jira/browse/HBASE-21014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573153#comment-16573153
]
Hari Sekhon edited comment on HBASE-21014 at 8/8/18 12:44 PM:
--------------------------------------------------------------
I thought that was the whole point of the FavoredNodeLoadBalancer - so that
HBase Balancer can write those HDFS hints based on the knowledge it has of
region locations so that HDFS Balancer can read the preferred location hints
and not move those blocks, therefore not losing data locality?
Normally I would just balance and then major compact but there are 2 issues
with running major compaction:
# performance impact - this cluster is production and heavily loaded
# this cluster is already running around 70-80% full which combined with HBase
rolling snapshots covering 4 days means that more than the one scheduled major
compaction a week would space exhaustion resulting in an outage (this very
nearly happened the first time I ran major compaction on this cluster but I
realised what was going on and took quick action to avoid an outage -
annoyingly there is not yet a major compaction cancel command in this version
of HBase)
was (Author: harisekhon):
I thought that was the whole point of the FavoredNodeLoadBalancer - so that
HBase Balancer can write those HDFS hints based on the knowledge it has of
region locations so that HDFS Balancer can read the preferred location hints
and not move those blocks, therefore not losing data locality?
> Improve Stochastic Balancer to write HDFS favoured node hints for region
> primary blocks to avoid destroying data locality if needing to use HDFS
> Balancer
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-21014
> URL: https://issues.apache.org/jira/browse/HBASE-21014
> Project: HBase
> Issue Type: Improvement
> Components: Balancer
> Affects Versions: 1.1.2
> Reporter: Hari Sekhon
> Priority: Major
>
> Improve Stochastic Balancer to include the HDFS region location hints to
> avoid HDFS Balancer destroying data locality.
> Right now according to a mix of docs, jiras and mailing list info it appears
> that one must change
> {code:java}
> hbase.master.loadbalancer.class{code}
> to the org.apache.hadoop.hbase.favored.FavoredNodeLoadBalancer as it looks
> like this functionality is only within FavoredNodeBalancer and not the
> standard Stochastic Balancer.
> [http://hbase.apache.org/book.html#_hbase_and_hdfs]
> This is not ideal because we'd still like to use all the heuristics and work
> that has gone in the Stochastic Balancer which I believe right now is the
> best and most mature HBase balancer.
> See also the linked Jiras and this discussion:
> [http://apache-hbase.679495.n3.nabble.com/HDFS-Balancer-td4086607.html]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
