[
https://issues.apache.org/jira/browse/HBASE-2699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886403#action_12886403
]
Jonathan Gray commented on HBASE-2699:
--------------------------------------
The load balancer will be triggered according to some configurable period, like
every 5 minutes (if you don't mind more churn) or once every hour (to keep
churn down).
The first thing it does is check that the cluster is in fact "out of balance",
and without slop this is strict. We used to not have slop but it was
introduced to prevent load balancing from getting triggered after just one or
two splits threw things just off balance. As a periodic background process, we
won't be sensitive to things like this and instead will do the balancing on a
fixed period.
There could be some improvements later that force a load balance if the cluster
reaches a certain level of "out of balance" but that's an optimization, not
considering it now.
This balancer should eventually move towards something cost-based that takes a
much more complete picture of "load" into account. I'm not sure a notion of
"slop" makes sense in this context... the goal will always be to balance as
best as possible (with a side-goal of not moving regions around if it doesn't
improve the balance... meaning, if a region could be on X or Y server and
achieve the same balance, it should bias towards staying where it is, minimal
churn). The algo will certainly need an overhaul when we implement this.
In any case, slop adds unnecessary complexity and the original purpose it was
added is no longer relevant IMO. With this new balancer, it would mean
overloaded servers would always hover at the top of the slop range, underloaded
servers would hover at the bottom of the slop range. Why not actually balance?
Simpler algorithm.
I also expect region movement to be quite fast with the double-flush and
zk-based transitions. Hopefully have some real timings next week.
> Reimplement load balancing to be a background process and to not use
> heartbeats
> -------------------------------------------------------------------------------
>
> Key: HBASE-2699
> URL: https://issues.apache.org/jira/browse/HBASE-2699
> Project: HBase
> Issue Type: Sub-task
> Components: master, regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Critical
> Fix For: 0.21.0
>
> Attachments: HBASE-2699-LoadBalancer-v2.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
