[
https://issues.apache.org/jira/browse/FLINK-19556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211527#comment-17211527
]
Xintong Song commented on FLINK-19556:
--------------------------------------
I think [~karmagyz] is correct.
The current implementation cannot support containers with different resources,
because (as described in FLINK-19324) Hadoop 2.8 and previous versions do not
support requesting containers of different resources with the same priority. To
support containers with different resources, we need to request them with
different priorities anyway. Therefore, using priority for identifying
containers with different resources would be much simpler than the current
resource matching, and we won't have problem with
\{{AMRMClientAsync.getMatchingRequests}} because the priority is strictly
matched.
The only concern I see in using priority for mapping requested/allocated
containers is that, we might enforce some sort of prioritization over the
containers. IMO, this is a necessary price for supporting containers with
different resources on Hadoop 2.8 and below. We can discuss this in FLINK-19324.
WDYT, [~trohrmann]?
> YarnResourceManagerDriver.getPendingRequestsAndCheckConsistency only returns
> container requests for first matching equivalent resource
> --------------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-19556
> URL: https://issues.apache.org/jira/browse/FLINK-19556
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN
> Affects Versions: 1.12.0
> Reporter: Till Rohrmann
> Priority: Major
> Fix For: 1.12.0
>
>
> The {{YarnResourceManagerDriver}} will retrieve the number of pending
> container requests for a given {{Resource}} by calling
> {{YarnResourceManagerDriver.getPendingRequestsAndCheckConsistency(Resource)}}.
> Part of this method is to find the equivalent resources for the given
> resource. Next we find for every equivalent resource the matching container
> requests. Instead of unifying these collection of container requests, the
> method returns the first non-empty collection. This might return an
> incomplete set of container requests if the Yarn respects the vcores of
> resources given to {{AMRMClientAsync.getMatchingRequests}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
