Junping Du created HADOOP-8470:
----------------------------------
Summary: Implementation of 4-layer subclass of NetworkTopology
(NetworkTopologyWithNodeGroup)
Key: HADOOP-8470
URL: https://issues.apache.org/jira/browse/HADOOP-8470
Project: Hadoop Common
Issue Type: Sub-task
Affects Versions: 2.0.0-alpha, 1.0.0
Reporter: Junping Du
Assignee: Junping Du
To support the four-layer hierarchical topology shown in attached figure as a
subclass of NetworkTopology, NetworkTopologyWithNodeGroup was developed along
with unit tests. NetworkTopologyWithNodeGroup overriding the methods add,
remove, and pseudoSortByDistance were the most relevant to support the
four-layer topology. The method seudoSortByDistance selects the nodes to use
for reading data and sorts the nodes in sequence of node-local,
nodegroup-local, rack- local, rack–off. Another slightly change to
seudoSortByDistance is to support cases of separation data node and node
manager: if the reader cannot be found in NetworkTopology tree (formed by data
nodes only), then it will try to sort according to reader's sibling node in the
tree.
The distance calculation changes the weights from 0 (local), 2 (rack- local), 4
(rack-off) to: 0 (local), 2 (nodegroup-local), 4 (rack-local), 6 (rack-off).
The additional node group layer should be specified in the topology script or
table mapping, e.g. input 10.1.1.1, output: /rack1/nodegroup1
A subclass on InnerNode, InnerNodeWithNodeGroup, was also needed to support
NetworkTopologyWithNodeGroup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
