[
https://issues.apache.org/jira/browse/HDFS-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15967462#comment-15967462
]
Yiqun Lin edited comment on HDFS-11530 at 4/13/17 11:44 AM:
------------------------------------------------------------
Thanks [~arpitagarwal] for the suggestion. The comment makes sense to me. The
followings are what I did in my latest patch and will help you review.
* Add the new setting {{dfs.net.topology.impl}} that only used to choose nodes
to place replicas but not used to balance/move block replicas.
* The default topology of the setting is {{NetworkTopology}} since we still
need more testings to test {{DFSNetworkTopology}} before making it as the
default(as [~arpitagarwal] suggested).
* Update place that used the old config name from
{{CommonConfigurationKeysPublic.NET_TOPOLOGY_IMPL_KEY}} to
{{DFSConfigKeys.DFS_NET_TOPOLOGY_IMPL_KEY}} and add an additional test to test
the function of the new DFSNetworkTopology which parsed from the new setting.
* Please look the v009 patch that fix some minors based on v008 patch.
Any comments are welcomed. Thanks.
was (Author: linyiqun):
Thanks [~arpitagarwal] for the suggestion. The comment makes sense to me. The
followings are what I did in my latest patch and will help you review.
* Add the new setting {{dfs.net.topology.impl}} that only used to choose nodes
to place replicas but not used to balance/move block replicas.
* The default topology of the setting is {{NetworkTopology}} since we still
need more testings to test {{DFSNetworkTopology}} before making it as the
default(as [~arpitagarwal] suggested).
* Update place that used the old config name from
{{CommonConfigurationKeysPublic.NET_TOPOLOGY_IMPL_KEY}} to
{{DFSConfigKeys.DFS_NET_TOPOLOGY_IMPL_KEY}} and add an additional test to test
the function of the new DFSNetworkTopology which parsed from the new setting.
Any comments are welcomed. Thanks.
> Use HDFS specific network topology to choose datanode in
> BlockPlacementPolicyDefault
> ------------------------------------------------------------------------------------
>
> Key: HDFS-11530
> URL: https://issues.apache.org/jira/browse/HDFS-11530
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Affects Versions: 3.0.0-alpha2
> Reporter: Yiqun Lin
> Assignee: Yiqun Lin
> Attachments: HDFS-11530.001.patch, HDFS-11530.002.patch,
> HDFS-11530.003.patch, HDFS-11530.004.patch, HDFS-11530.005.patch,
> HDFS-11530.006.patch, HDFS-11530.007.patch, HDFS-11530.008.patch,
> HDFS-11530.009.patch, HDFS-11530.010.patch, HDFS-11530.011.patch
>
>
> The work for {{chooseRandomWithStorageType}} has been merged in HDFS-11482.
> But this method is contained in new topology {{DFSNetworkTopology}} which is
> specified for HDFS. We should update this and let
> {{BlockPlacementPolicyDefault}} use the new way since the original way is
> inefficient.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
