[
https://issues.apache.org/jira/browse/HDFS-10208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-10208:
---------------------------
Attachment: HDFS-10208-3.patch
Thanks [~brahmareddy]. I have updated the property description in
core-default.xml.
>From the investigation of HDFS-10206, the current way of using Topology tree
>to compute node distance by reference seems too expensive. It means it needs
>to add nodes to the Topology tree first which could become slow as the tree
>grows. It also means the tree size could grow unbounded. To solve this issue,
>we can use network path string comparison instead without Topology tree and
>the extra HashMap in ClientContext.
So in summary, the patch has three improvements:
* Handle the case the client can't resolve network path properly.
* Make the client-side topology resolution optional.
* Use string based comparison for network distance calculation.
> Addendum for HDFS-9579: to handle the case when client machine can't resolve
> network path
> -----------------------------------------------------------------------------------------
>
> Key: HDFS-10208
> URL: https://issues.apache.org/jira/browse/HDFS-10208
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Ming Ma
> Assignee: Ming Ma
> Attachments: HDFS-10208-2.patch, HDFS-10208-3.patch, HDFS-10208.patch
>
>
> If DFSClient runs on a machine that can't resolve network path,
> e.g.{{dnsToSwitchMapping.resolve}} returns null, that will cause exception
> when it tries to create {{clientNode}}. In such case, there is no need to
> create {{clientNode}} as null {{clientNode}} means its network distance with
> any datanode is Integer.MAX_VALUE, which is what we want.
> {noformat}
> clientNode = new NodeBase(clientHostName,
> dnsToSwitchMapping.resolve(nodes).get(0));
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
