[ https://issues.apache.org/jira/browse/HDFS-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16433619#comment-16433619 ]
Yiqun Lin edited comment on HDFS-13418 at 4/11/18 9:37 AM: ----------------------------------------------------------- Thanks [~Tao Jie] for updating the patch. Almost looks good, some comments from me: * Can we define a var {{DFS_NET_TOPOLOGY_IMPL_DEFAULT}} as the default NetworkTopology impl class used in HDFS rather than hard-coded in {{DFSNetworkTopology#getInstance}}. * For the description of {{dfs.net.topology.impl}}, {{The implementation class of NetworkTopology used in block placement.}} can be updated to {{The implementation class of NetworkTopology used in HDFS.}} And then adding new one sentence to introduce the default impl class: {{By default, the class org.apache.hadoop.hdfs.net.DFSNetworkTopology is specified and used in block placement.}} For the unit test, seems we don't need to extend the new NetworkTopology to test. Current existing {{NetworkTopology}} impls are enough. I think there will be three cases we need to cover. * Switch {{dfs.use.dfs.network.topology}} to true, then verify if the default DFSNetworkTopology impl is got. * Switch {{dfs.use.dfs.network.topology}} to false, then verify if the default NetworkTopology impl is got. * Switch {{dfs.use.dfs.network.topology}} to false, and change the value of {{net.topology.impl}}, then verify if the desired NetworkTopology impl is got. Would you mind make a minor adjustment for addressing these three cases? was (Author: linyiqun): Thanks [~Tao Jie] for updating the patch. Almost looks good, some comments from me: * Can we define a var {{DFS_NET_TOPOLOGY_IMPL_DEFAULT}} as the default NetworkTopology impl class used in HDFS rather than hard-coded in {{DFSNetworkTopology#getInstance}}. * For the description of {{dfs.net.topology.impl}}, {{The implementation class of NetworkTopology used in block placement.}} can be updated to {{The implementation class of NetworkTopology used in HDFS.}} And then adding new one sentence to introduce the default impl class: {{By default, the class org.apache.hadoop.hdfs.net.DFSNetworkTopology is specified and used in block placement.}} For the unit test, seems we don't need to extend the new NetworkTopology to test. Current existing {{NetworkTopology}} impls are enough. I think there will be three cases we need to cover. * Switch {{dfs.use.dfs.network.topology}} to true, then verify if the default DFSNetworkTopology impl is got. * Switch {{dfs.use.dfs.network.topology}} to false, then verify if the default NetworkTopology impl is got. * Switch {{dfs.use.dfs.network.topology}} to false, and change the value of {{net.topology.impl}}, then verify if the desired NetworkTopology impl is got. > NetworkTopology should be configurable when enable DFSNetworkTopology > ---------------------------------------------------------------------- > > Key: HDFS-13418 > URL: https://issues.apache.org/jira/browse/HDFS-13418 > Project: Hadoop HDFS > Issue Type: Improvement > Affects Versions: 3.0.1 > Reporter: Tao Jie > Assignee: Tao Jie > Priority: Major > Attachments: HDFS-13418.001.patch, HDFS-13418.002.patch > > > In HDFS-11530 we introduce DFSNetworkTopology and in HDFS-11998 we set > DFSNetworkTopology as the default implementation. > We still have {{net.topology.impl=org.apache.hadoop.net.NetworkTopology}} in > core-site.default. Actually this property does not effect once > {{dfs.use.dfs.network.topology}} is true. > in {{DatanodeManager}},networkTopology is initialized as > {code} > if (useDfsNetworkTopology) { > networktopology = DFSNetworkTopology.getInstance(conf); > } else { > networktopology = NetworkTopology.getInstance(conf); > } > {code} > I think we should still make the NetworkTopology configurable rather than > hard code the implementation since we may need another NetworkTopology impl. > I am not sure if there is other consideration. Any thought? [~vagarychen] > [~linyiqun] -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org