Github user shuai-xu commented on a diff in the pull request:
https://github.com/apache/flink/pull/3304#discussion_r102907061
--- Diff:
flink-yarn/src/main/java/org/apache/flink/yarn/YarnResourceManager.java ---
@@ -376,4 +388,23 @@ private int generatePriority(ResourceProfile
resourceProfile) {
}
}
+ /**
+ * Get resource profile by given the container priority. It should
always be got from the mapping
+ * between resource profile and priority, otherwise it will throw the
exception.
+ *
+ * @param priority The container priority used for distinguishing
different resources.
+ * @return The resource profile corresponding with the priority.
+ * @throws Exception
+ */
+ private ResourceProfile getResourceProfile(int priority) throws
Exception{
+ if (resourcePriorities.containsValue(priority)) {
+ for (Map.Entry<ResourceProfile, Integer> entry :
resourcePriorities.entrySet()) {
+ if (entry.getValue() == priority) {
+ return entry.getKey();
+ }
+ }
+ }
--- End diff --
This is a confusing part in yarn. Now the priority is only used to
distinguish requests of different resource. It has nothing to do with
importance. For slot request of different resources, we should generate
different priorities for them, and put priority and resource profile to the
resourcePriorities map. When yarn containers come back, we know they should
start the task manager with which resource profile by query the map with the
container priority. And when task manager offer slot with the resource profile
to job master, we can exactly match the slot with the original request.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
