[
https://issues.apache.org/jira/browse/HDFS-10243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15221779#comment-15221779
]
Vinayakumar B commented on HDFS-10243:
--------------------------------------
{{dnsToSwitchMapping.reloadCachedMappings(invalidNodeNames);}} will be called
only when the resolved values are at different levels.
In your case, already exist nodes have 2 level mapping. switch and rack. But
new node is getting resolved to default value of just rack level. So
{{InvalidTopologyException}} will come, in which case. Patch proposed will
reload the contents.
But, if original mappings in the file are also at 1 level (rack), then patch
will have no effect.
How about loading the file, during resolve itself?
Below changes will reload everytime a unresolved and uncached node is trying to
resolve.
{code}
@@ -133,14 +137,22 @@ public void reloadCachedMappings() {
map = new HashMap<String, String>();
}
}
+ boolean justLoaded = false;
List<String> results = new ArrayList<String>(names.size());
for (String name : names) {
String result = map.get(name);
- if (result != null) {
- results.add(result);
- } else {
- results.add(NetworkTopology.DEFAULT_RACK);
+ if (result == null) {
+ // try to re-load map, if not recently loaded.
+ if (!justLoaded) {
+ map = load();
+ justLoaded = true;
+ result = map.get(name);
+ }
+ if (result == null) {
+ result = NetworkTopology.DEFAULT_RACK;
+ }
}
+ results.add(result);
}
return results;
}
{code}
> When using the TableMapping network topology, adding new datanoe need to
> restart the namenode
> ---------------------------------------------------------------------------------------------
>
> Key: HDFS-10243
> URL: https://issues.apache.org/jira/browse/HDFS-10243
> Project: Hadoop HDFS
> Issue Type: Bug
> Environment: 2.6.0
> Reporter: shenxianqiang
> Priority: Minor
> Attachments: HDFS-10243.patch
>
>
> When I use the TableMapping network topology, adding new machines need to
> restart the namenode. Configure :
> {quote}
> <property>
> <name>net.topology.node.switch.mapping.impl</name>
> <value>org.apache.hadoop.net.TableMapping</value>
> </property>
> <property>
> <name>net.topology.table.file.name</name>
> <value>/etc/hadoop/conf/net_topology_table</value>
> </property>
> {quote}
> In /etc/hadoop/conf/net_topology_table:
> {quote}
> 10.0.0.1 /SJS/SJS-1
> 10.0.0.2 /CTC/CTC-2
> 10.0.0.3 /TC/TC-3
> {quote}
> When I add a new datanode like 10.0.0.4 /SJS/SJS-2,datanode throw exception:
> {quote}
> 2016-03-30 17:11:15,608 ERROR
> org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for
> Block pool BP-408802935-10.0.0.100-1402130194887 (Datanode Uuid null) service
> to nn1/10.0.0.100:8020 Failed to add /default-rack/10.0.0.4:50010: You cannot
> have a rack and a non-rack node at the same level of the network topology.
> at org.apache.hadoop.net.NetworkTopology.add(NetworkTopology.java:408)
> at
> org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.registerDatanode(DatanodeManager.java:1001)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.registerDatanode(FSNamesystem.java:4837)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.registerDatanode(NameNodeRpcServer.java:1038)
> at
> org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.registerDatanode(DatanodeProtocolServerSideTranslatorPB.java:92)
> at
> org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:26378)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:587)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1026)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1892)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
> {quote}
> I need to update /etc/hadoop/conf/net_topology_table,and restart
> namenode.After that,the new datanode should work.
> Restart Namenode may cause a bad influence.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
