[
https://issues.apache.org/jira/browse/YARN-6406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15951542#comment-15951542
]
Jason Lowe commented on YARN-6406:
----------------------------------
Yep, the refcount was only added because of the possibility of the two types of
requests. When there are multiple refs to the key, we can't assume removing
the last of one type removes all references to the key. If there is only one
type that can reference the scheduler key then we don't need to refcount it
separately.
> Garbage Collect unused SchedulerRequestKeys
> -------------------------------------------
>
> Key: YARN-6406
> URL: https://issues.apache.org/jira/browse/YARN-6406
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Arun Suresh
> Assignee: Arun Suresh
>
> YARN-5540 introduced some optimizations to remove satisfied SchedulerKeys
> from the AppScheduleingInfo. It looks like after YARN-6040,
> ScedulerRequestKeys are removed only if the Application sends a 0
> numContainers requests. While earlier, the outstanding schedulerKeys were
> also remove as soon as a container is allocated as well.
> An additional optimization we were hoping to include is to remove the
> ResourceRequests itself once the numContainers == 0, since we see in our
> clusters that the RM heap space consumption increases drastically due to a
> large number of ResourceRequests with 0 num containers.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
