ChenSammi commented on a change in pull request #1194: HDDS-1879.  Support 
multiple excluded scopes when choosing datanodes in NetworkTopology
URL: https://github.com/apache/hadoop/pull/1194#discussion_r322684290
 
 

 ##########
 File path: 
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/net/NetworkTopologyImpl.java
 ##########
 @@ -509,40 +516,48 @@ private Node chooseNodeInternal(String scope, int 
leafIndex,
       ancestorGen = 0;
     }
 
-    // check overlap of excludedScope and finalScope
-    if (excludedScope != null) {
-      // excludeScope covers finalScope
-      if (finalScope.startsWith(excludedScope)) {
-        return null;
-      }
-      // excludeScope and finalScope share nothing
-      if (!excludedScope.startsWith(finalScope)) {
-        excludedScope = null;
+    // check overlap of excludedScopes and finalScope
+    List<String> mutableExcludedScopes = null;
+    if (excludedScopes != null && !excludedScopes.isEmpty()) {
+      mutableExcludedScopes = new ArrayList<>();
+      for (String s: excludedScopes) {
+        // excludeScope covers finalScope
+        if (finalScope.startsWith(s)) {
+          return null;
+        }
+        // excludeScope and finalScope share nothing case
+        if (s.startsWith(finalScope)) {
+          if (!mutableExcludedScopes.stream().anyMatch(
+              e -> s.startsWith(e))) {
+            mutableExcludedScopes.add(s);
+          }
+        }
       }
     }
 
     // clone excludedNodes before remove duplicate in it
     Collection<Node> mutableExNodes = null;
+
+    // Remove duplicate in excludedNodes
     if (excludedNodes != null) {
-      // Remove duplicate in excludedNodes
       mutableExNodes =
           excludedNodes.stream().distinct().collect(Collectors.toList());
     }
 
-    // remove duplicate in mutableExNodes and excludedScope, given ancestorGen
-    excludedScope = NetUtils.removeDuplicate(this, mutableExNodes,
-        excludedScope, ancestorGen);
+    // remove duplicate in mutableExNodes and mutableExcludedScopes
+    NetUtils.removeDuplicate(this, mutableExNodes, mutableExcludedScopes,
+        ancestorGen);
 
     // calculate available node count
     Node scopeNode = getNode(finalScope);
     int availableNodes = getAvailableNodesCount(
-        scopeNode.getNetworkFullPath(), excludedScope, mutableExNodes,
+        scopeNode.getNetworkFullPath(), mutableExcludedScopes, mutableExNodes,
         ancestorGen);
 
     if (availableNodes <= 0) {
       LOG.warn("No available node in (scope=\"{}\" excludedScope=\"{}\" " +
 
 Review comment:
   Thanks for point out.  I double checked the output, the list content is 
printed instead of list address.  See this log, 
   
   2019-09-10 19:14:28,029 WARN  net.NetworkTopology 
(NetworkTopologyImpl.java:chooseNodeInternal(538)) - No available node in 
(scope="/" excludedScope="[/rack0/node2]" 
excludedNodes="[7ba208c8-7ef9-4ec0-abaf-7cd71953f43b{ip: 2.124.202.116, host: 
node11, networkLocation: /rack2, certSerialId: null}, 
953a7a44-abd1-4d51-ae0b-b1d641f5ce5a{ip: 188.152.123.3, host: node10, 
networkLocation: /rack2, certSerialId: null}, 
853c712b-aa56-43f5-a1ac-8b101bbf9f1d{ip: 232.74.9.147, host: node9, 
networkLocation: /rack1, certSerialId: null}]"  ancestorGen="1").

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

Reply via email to