Junping Du created HADOOP-8473:
----------------------------------
Summary: Update Balancer to support new NetworkTopology with
NodeGroup
Key: HADOOP-8473
URL: https://issues.apache.org/jira/browse/HADOOP-8473
Project: Hadoop Common
Issue Type: Sub-task
Components: util
Affects Versions: 2.0.0-alpha, 1.0.0
Reporter: Junping Du
Assignee: Junping Du
Since the Balancer is a Hadoop Tool, it was updated to be directly aware of
four-layer hierarchy instead of creating an alternative Balancer
implementation. To accommodate extensibility, a new protected method,
doChooseNodesForCustomFaultDomain is now called from the existing chooseNodes
method so that a subclass of the Balancer could customize the balancer
algotirhm for other failure and locality topologies. An alternative option is
to encapsulate the algorithm used for the four-layer hierarchy into a
collaborating strategy class.
The key changes introduced to support a four-layer hierarchy were to override
the algorithm of choosing <source, target> pairs for balancing. Unit tests were
created to test the new algorithm.
The algorithm now makes sure to choose the target and source node on the same
node group for balancing as the first priority. Then the overall balancing
policy is: first doing balancing between nodes within the same nodegroup then
the same rack and off rack at last. Also, we need to check no duplicated
replicas live in the same node group after balancing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
