[
https://issues.apache.org/jira/browse/HBASE-2699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Gray updated HBASE-2699:
---------------------------------
Attachment: HBASE-2699-LoadBalancer-v2.patch
Adds new {{LoadBalancer}} class to the master package. Class defines three
primary methods.
The new load balancing algorithm drops the notion of slop and fully balances
the cluster (this will be a background process so no need to use slop to
prevent churn). The method is a bit long but I did my best to document it...
at least it's in one method and testable now :) Balancing uses the following
method which takes a map of all the servers to their regions and returns a list
of regions to be moved with their source and destination servers:
{noformat}
public List<RegionPlan> balanceCluster(Map<HServerInfo,List<HRegionInfo>>
serverInfo)
{noformat}
For cluster startup, there is a bulk assignment method. Currently this is
implemented as round-robin but should eventually be extended to use block
locations (there are some methods in this class to help but not integrated yet):
{noformat}
public Map<HServerInfo,List<HRegionInfo>> bulkAssignment(List<HRegionInfo>
regions, List<HServerInfo> servers)
{noformat}
The other time we may need to determine assignments of regions is after master
failover. A new master may find regions who have been closed but no longer has
in-memory state (a {{RegionPlan}}) to say where it should be opened. In this
case, the master will use an immediate assignment method, the goal of which is
to just come up with an assignment as fast as possible. Currently it randomly
picks a server for each input region.
{noformat}
public Map<HRegionInfo,HServerInfo> immediateAssignment(List<HRegionInfo>
regions, List<HServerInfo> servers)
{noformat}
Patch also adds a nice fat TestLoadBalancer unit test with tests for each of
the three methods. It has input arrays for a bunch of mock cluster states and
methods to verify the balancing invariants to ensure correctness.
If anyone has more good edge cases let's get em added!
Yay, a testable load balancer!
> Reimplement load balancing to be a background process and to not use
> heartbeats
> -------------------------------------------------------------------------------
>
> Key: HBASE-2699
> URL: https://issues.apache.org/jira/browse/HBASE-2699
> Project: HBase
> Issue Type: Sub-task
> Components: master, regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Priority: Critical
> Fix For: 0.21.0
>
> Attachments: HBASE-2699-LoadBalancer-v2.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
