[
https://issues.apache.org/jira/browse/HADOOP-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12564858#action_12564858
]
Mark Butler commented on HADOOP-2767:
-------------------------------------
Further examination shows the problem only occurs when you are excluding a
node, not when you are excluding a rack. So it may not be a problem for DFS ...
However, if it does occur, it is not just a problem of not using a node, it
also causes another node to be a "hotspot" as it is twice as likely as any
other node to get a replica placement., which is clearly undesirable ..
Can someone more familiar with DFS replication code comment if this is likely
to be a problem?
> org.apache.hadoop.net.NetworkTopology.InnerNode#getLeaf does not return the
> last node on a rack when used with an excluded node
> -------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-2767
> URL: https://issues.apache.org/jira/browse/HADOOP-2767
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: hudson
> Reporter: Mark Butler
> Priority: Minor
> Fix For: hudson
>
> Attachments: NetworkTopologyTest.java
>
>
> I have written some test code that shows NetworkTopology.InnerNode#getLeaf
> will never return the last node on the rack if it is called with an
> excludedNode (for example the first node on the rack).
> Consequently I suspect that NetworkTopology.chooseRandom() will never returns
> the last node on the remote rack for the second replica in DFS.
> I have some test code that demonstrates this problem at the getLeaf level,
> although it is necessary to change the visibility of the
> NetworkTopology.InnerNode, NetworkTopology.InnerNode#getLeaf and
> NetworkTopology.getNode from private to package default to run the test.
> TODO: Demonstrate problem at NetworkTopology.chooseRandom level, then submit
> candidate fix.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
