[
https://issues.apache.org/jira/browse/HDFS-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431918#comment-16431918
]
Yiqun Lin commented on HDFS-13418:
----------------------------------
Hi [~Tao Jie],
{quote}1, Can we just remove
net.topology.impl=org.apache.hadoop.net.NetworkTopology in core-default.xml?
Since we have default impl in code.
{quote}
I don't agree on this. NetworkTopology is under COMMON and won't be only used
for HDFS in the future.
{quote}2, Can we set net.topology.impl=org.apache.hadoop.net.DFSNetworkTopology
in hdfs-default.xml? So it would cover the value set in core-site.default when
we use HDFS.
{quote}
Setting {{net.topology.impl=org.apache.hadoop.net.DFSNetworkTopology}} will not
make sense. In current logic, {{NetworkTopology#getInstance}} cannot construct
{{DFSNetworkTopology}} instance.
{quote}3, If we want to add a new NetworkTopology impl and work with
DFSNetworkTopology, we may define the new NetworkTopology extends
DFSNetworkTopology. But it does not work since DFSNetworkTopology is hardcoded
here once dfs.use.dfs.network.topology is true. Can we use reflection to
instantiate the NetworkTopology impl when dfs.use.dfs.network.topology is set
true?
{quote}
Actually we only use {{DFSNetworkTopology}} for block placement, not mean the
{{NetworkTopology}} never be used. For the Balancer dispatcher, we are still
using {{NetworkTopology}} that we are configured. Yes, you have mentioned one
point that we can use {{NetworkTopology}} to reflect {{DFSNetworkTopology}} and
reuse the {{net.topology.impl}}. So this will look better. But
{{DFSNetworkTopology}} is only making sense to HDFS, put it in COMMON won't a
good way I think. Also it will not be convenient to construct since it needs
some dependency of HDFS.
> NetworkTopology should be configurable when enable DFSNetworkTopology
> ----------------------------------------------------------------------
>
> Key: HDFS-13418
> URL: https://issues.apache.org/jira/browse/HDFS-13418
> Project: Hadoop HDFS
> Issue Type: Improvement
> Affects Versions: 3.0.1
> Reporter: Tao Jie
> Assignee: Tao Jie
> Priority: Major
>
> In HDFS-11530 we introduce DFSNetworkTopology and in HDFS-11998 we set
> DFSNetworkTopology as the default implementation.
> We still have {{net.topology.impl=org.apache.hadoop.net.NetworkTopology}} in
> core-site.default. Actually this property does not effect once
> {{dfs.use.dfs.network.topology}} is true.
> in {{DatanodeManager}}，networkTopology is initialized as
> {code}
> if (useDfsNetworkTopology) {
> networktopology = DFSNetworkTopology.getInstance(conf);
> } else {
> networktopology = NetworkTopology.getInstance(conf);
> }
> {code}
> I think we should still make the NetworkTopology configurable rather than
> hard code the implementation since we may need another NetworkTopology impl.
> I am not sure if there is other consideration. Any thought? [~vagarychen]
> [~linyiqun]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
