[
https://issues.apache.org/jira/browse/YARN-6406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15951845#comment-15951845
]
Arun Suresh commented on YARN-6406:
-----------------------------------
Thanks for the review [~leftnoteasy],
bq. Why changes of AppInfo required?
Hmmm.. so the the {{TestRMWebServicesApps}} was complaining since it needed a
resource request object. With this patch, if there is no outstanding resource
request, The AppInfo will not contain any resource request objects. So I
decided to send a dummy resourceRequest Object when none exist. I think the
right approach is to fix the test case (which might be harder).. thoughts ?
bq. In LocalitySchedulingPlacementSet: it calls appSchedulingInfo directly in
decrementOutstanding ...
Don't think it is a problem too much (based on existing code paths) .. but yes,
maybe we should clean it up, since it could lead some circular references to
the same placementset object.
Please feel free to open another JIRA for that (I can help review), but for the
timebeing, I think we can remove the schedulerkey as is done in this patch ?
> Garbage Collect unused SchedulerRequestKeys
> -------------------------------------------
>
> Key: YARN-6406
> URL: https://issues.apache.org/jira/browse/YARN-6406
> Project: Hadoop YARN
> Issue Type: Improvement
> Affects Versions: 2.8.0, 2.7.3, 3.0.0-alpha2
> Reporter: Arun Suresh
> Assignee: Arun Suresh
> Attachments: YARN-6406.001.patch
>
>
> YARN-5540 introduced some optimizations to remove satisfied SchedulerKeys
> from the AppScheduleingInfo. It looks like after YARN-6040,
> ScedulerRequestKeys are removed only if the Application sends a 0
> numContainers requests. While earlier, the outstanding schedulerKeys were
> also remove as soon as a container is allocated as well.
> An additional optimization we were hoping to include is to remove the
> ResourceRequests itself once the numContainers == 0, since we see in our
> clusters that the RM heap space consumption increases drastically due to a
> large number of ResourceRequests with 0 num containers.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
