[
https://issues.apache.org/jira/browse/YARN-5353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated YARN-5353:
-----------------------------
Attachment: YARN-5353.001.patch
Seems to me that we need to make sure that the appTokens map always has the
application removed when the application is marked as finished. It's our one
chance to clean up the app entry, and currently the code can conditionally
decide to leave the app's entry in the map.
Attaching a patch that always removes the appTokens entry corresponding to an
app when the app finished event is received. Any tokens that are shared with
other apps will continue to exist in the allTokens map, so I think we'll still
be good as far as token-sharing goes.
> ResourceManager can leak delegation tokens when they are shared across apps
> ---------------------------------------------------------------------------
>
> Key: YARN-5353
> URL: https://issues.apache.org/jira/browse/YARN-5353
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
> Affects Versions: 2.7.0, 2.6.1
> Reporter: Jason Lowe
> Priority: Critical
> Attachments: YARN-5353.001.patch
>
>
> Recently saw a ResourceManager go into heavy GC. Heap dump showed that there
> were millions of delegation tokens on the heap. It looks like most of them
> belonged to the appTokens map in DelegationTokenRenewer. When an app
> completes and tokens are removed for it, I noticed that the appTokens entry
> for the app is not cleaned up if tokens were shared with other active apps.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
