[
https://issues.apache.org/jira/browse/HADOOP-8470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393428#comment-13393428
]
Junping Du commented on HADOOP-8470:
------------------------------------
Hey Nicholas,
Thanks for review and comments.
isSameParents() should be reused for isOnSameRack and isOnSameNodeGroup. It
is my mistake that not using it in isOnSameNodeGroup which confuse you. Shall
we update in isOnSameNodeGroup to use this api?
isNodeGroupAware() will be used in balancer policy change
(https://issues.apache.org/jira/browse/HDFS-3495). It is not a must existing
API in functional but without it, the balancer will need to use reflection to
judge if a NetworkTopology object belongs to NetworkTopologyWithNodeGroup
class. We don't want "NetworkTopologyWithNodeGroup" to show up too much in the
glue code of rest classes. Isn't it? Shall we delay this API going in until we
go to balancer or some tests of replication policy? but we already have this
API in NetworkTopology in trunk. Thoughts?
InnerNodeWithNodeGroup(String name, String location) constructor is not
used. Yes. we should remove it as well as the same api in its parent class
(InnerNode). What do you think?
> Implementation of 4-layer subclass of NetworkTopology
> (NetworkTopologyWithNodeGroup)
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-8470
> URL: https://issues.apache.org/jira/browse/HADOOP-8470
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 1.0.0, 2.0.0-alpha
> Reporter: Junping Du
> Assignee: Junping Du
> Attachments: HADOOP-8470-NetworkTopology-new-impl-v2.patch,
> HADOOP-8470-NetworkTopology-new-impl-v3.patch,
> HADOOP-8470-NetworkTopology-new-impl.patch
>
>
> To support the four-layer hierarchical topology shown in attached figure as a
> subclass of NetworkTopology, NetworkTopologyWithNodeGroup was developed along
> with unit tests. NetworkTopologyWithNodeGroup overriding the methods add,
> remove, and pseudoSortByDistance were the most relevant to support the
> four-layer topology. The method seudoSortByDistance selects the nodes to use
> for reading data and sorts the nodes in sequence of node-local,
> nodegroup-local, rack- local, rack–off. Another slightly change to
> seudoSortByDistance is to support cases of separation data node and node
> manager: if the reader cannot be found in NetworkTopology tree (formed by
> data nodes only), then it will try to sort according to reader's sibling node
> in the tree.
> The distance calculation changes the weights from 0 (local), 2 (rack- local),
> 4 (rack-off) to: 0 (local), 2 (nodegroup-local), 4 (rack-local), 6 (rack-off).
> The additional node group layer should be specified in the topology script or
> table mapping, e.g. input 10.1.1.1, output: /rack1/nodegroup1
> A subclass on InnerNode, InnerNodeWithNodeGroup, was also needed to support
> NetworkTopologyWithNodeGroup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
