homatthew commented on code in PR #3586:
URL: https://github.com/apache/gobblin/pull/3586#discussion_r1003565204
##########
gobblin-yarn/src/main/java/org/apache/gobblin/yarn/YarnService.java:
##########
@@ -889,11 +902,11 @@ public void onContainersAllocated(List<Container>
containers) {
allocatedContainerCountMap.putIfAbsent(containerHelixTag, new
AtomicInteger(0));
allocatedContainerCountMap.get(containerHelixTag).incrementAndGet();
- // Find matching requests and remove the request to reduce the chance
that a subsequent request
- // will request extra containers. YARN does not have a delta request
API and the requests are not
- // cleaned up automatically.
+ // Find matching requests and remove the request (YARN-660). We the
scheduler are responsible
Review Comment:
https://issues.apache.org/jira/browse/YARN-660?focusedCommentId=13655384&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13655384
> This jira takes up the problem of helping schedulers find matching
requests for allocated containers.
>
> ...
>
> Now the API becomes elegant and intuitive. addContainerRequest() to add
requests. getMatchingRequests() to get all matching requests. Pick a container
from matching requests and call removeContainerRequest() with it to remove it.
That is all that there is to it. There are some other minor fixes to
AMRMClientAsync. We are on the same page wrt this being a needed functionality.
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
