[
https://issues.apache.org/jira/browse/HDDS-1879?focusedWorklogId=309709&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-309709
]
ASF GitHub Bot logged work on HDDS-1879:
----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Sep/19 11:19
Start Date: 10/Sep/19 11:19
Worklog Time Spent: 10m
Work Description: ChenSammi commented on pull request #1194: HDDS-1879.
Support multiple excluded scopes when choosing datanodes in NetworkTopology
URL: https://github.com/apache/hadoop/pull/1194#discussion_r322684290
##########
File path:
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/net/NetworkTopologyImpl.java
##########
@@ -509,40 +516,48 @@ private Node chooseNodeInternal(String scope, int
leafIndex,
ancestorGen = 0;
}
- // check overlap of excludedScope and finalScope
- if (excludedScope != null) {
- // excludeScope covers finalScope
- if (finalScope.startsWith(excludedScope)) {
- return null;
- }
- // excludeScope and finalScope share nothing
- if (!excludedScope.startsWith(finalScope)) {
- excludedScope = null;
+ // check overlap of excludedScopes and finalScope
+ List<String> mutableExcludedScopes = null;
+ if (excludedScopes != null && !excludedScopes.isEmpty()) {
+ mutableExcludedScopes = new ArrayList<>();
+ for (String s: excludedScopes) {
+ // excludeScope covers finalScope
+ if (finalScope.startsWith(s)) {
+ return null;
+ }
+ // excludeScope and finalScope share nothing case
+ if (s.startsWith(finalScope)) {
+ if (!mutableExcludedScopes.stream().anyMatch(
+ e -> s.startsWith(e))) {
+ mutableExcludedScopes.add(s);
+ }
+ }
}
}
// clone excludedNodes before remove duplicate in it
Collection<Node> mutableExNodes = null;
+
+ // Remove duplicate in excludedNodes
if (excludedNodes != null) {
- // Remove duplicate in excludedNodes
mutableExNodes =
excludedNodes.stream().distinct().collect(Collectors.toList());
}
- // remove duplicate in mutableExNodes and excludedScope, given ancestorGen
- excludedScope = NetUtils.removeDuplicate(this, mutableExNodes,
- excludedScope, ancestorGen);
+ // remove duplicate in mutableExNodes and mutableExcludedScopes
+ NetUtils.removeDuplicate(this, mutableExNodes, mutableExcludedScopes,
+ ancestorGen);
// calculate available node count
Node scopeNode = getNode(finalScope);
int availableNodes = getAvailableNodesCount(
- scopeNode.getNetworkFullPath(), excludedScope, mutableExNodes,
+ scopeNode.getNetworkFullPath(), mutableExcludedScopes, mutableExNodes,
ancestorGen);
if (availableNodes <= 0) {
LOG.warn("No available node in (scope=\"{}\" excludedScope=\"{}\" " +
Review comment:
Thanks for point out. I double checked the output, the list content is
printed instead of list address. See this log,
2019-09-10 19:14:28,029 WARN net.NetworkTopology
(NetworkTopologyImpl.java:chooseNodeInternal(538)) - No available node in
(scope="/" excludedScope="[/rack0/node2]"
excludedNodes="[7ba208c8-7ef9-4ec0-abaf-7cd71953f43b{ip: 2.124.202.116, host:
node11, networkLocation: /rack2, certSerialId: null},
953a7a44-abd1-4d51-ae0b-b1d641f5ce5a{ip: 188.152.123.3, host: node10,
networkLocation: /rack2, certSerialId: null},
853c712b-aa56-43f5-a1ac-8b101bbf9f1d{ip: 232.74.9.147, host: node9,
networkLocation: /rack1, certSerialId: null}]" ancestorGen="1").
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 309709)
Time Spent: 3.5h (was: 3h 20m)
> Support multiple excluded scopes when choosing datanodes in NetworkTopology
> ---------------------------------------------------------------------------
>
> Key: HDDS-1879
> URL: https://issues.apache.org/jira/browse/HDDS-1879
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Reporter: Sammi Chen
> Assignee: Sammi Chen
> Priority: Major
> Labels: pull-request-available
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
