[
https://issues.apache.org/jira/browse/FLINK-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497943#comment-16497943
]
ASF GitHub Bot commented on FLINK-7866:
---------------------------------------
Github user sihuazhou commented on a diff in the pull request:
https://github.com/apache/flink/pull/4949#discussion_r192384844
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/types/SlotProfile.java
---
@@ -203,68 +206,104 @@ public
LocalityAwareRequirementsToSlotMatcher(@Nonnull Collection<TaskManagerLoc
}
// we build up two indexes, one for resource id and one
for host names of the preferred locations.
- HashSet<ResourceID> preferredResourceIDs = new
HashSet<>(locationPreferences.size());
- HashSet<String> preferredFQHostNames = new
HashSet<>(locationPreferences.size());
+ Map<ResourceID, Integer> preferredResourceIDs = new
HashMap<>(locationPreferences.size());
+ Map<String, Integer> preferredFQHostNames = new
HashMap<>(locationPreferences.size());
for (TaskManagerLocation locationPreference :
locationPreferences) {
-
preferredResourceIDs.add(locationPreference.getResourceID());
-
preferredFQHostNames.add(locationPreference.getFQDNHostname());
+ Integer oldVal =
preferredResourceIDs.getOrDefault(locationPreference.getResourceID(), 0);
+
preferredResourceIDs.put(locationPreference.getResourceID(), oldVal + 1);
+
+ oldVal =
preferredFQHostNames.getOrDefault(locationPreference.getFQDNHostname(), 0);
+
preferredFQHostNames.put(locationPreference.getFQDNHostname(), oldVal + 1);
}
Iterator<IN> iterator = candidates.iterator();
- IN matchByHostName = null;
IN matchByAdditionalRequirements = null;
+ final Map<IN, CandidateMatchedResult>
candidateMatchedResults = new HashMap<>();
--- End diff --
@StefanRRichter thanks for your review, I'm a bit forgot the logical of
this code now...I will take a look and think about your comments.
> Weigh list of preferred locations for scheduling
> ------------------------------------------------
>
> Key: FLINK-7866
> URL: https://issues.apache.org/jira/browse/FLINK-7866
> Project: Flink
> Issue Type: Improvement
> Components: Scheduler
> Affects Versions: 1.4.0, 1.3.2
> Reporter: Till Rohrmann
> Assignee: Sihua Zhou
> Priority: Major
> Fix For: 1.6.0
>
>
> [~sihuazhou] proposed to not only use the list of preferred locations to
> decide where to schedule a task, but to also weigh the list according to how
> often a location appeared and then select the location based on the weight.
> That way, we would obtain better locality in some cases.
> Example:
> Preferred locations list: {{[location1, location2, location2]}}
> Weighted preferred locations list {{[(location2 , 2), (location1, 1)]}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
