[
https://issues.apache.org/jira/browse/YARN-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16554751#comment-16554751
]
Suma Shivaprasad commented on YARN-8418:
----------------------------------------
Thanks for the clarification [~bibinchundatt]. A couple of more
questions/comments
1.
{noformat}
// Schedule the aggregator.
Runnable aggregatorWrapper = new Runnable() {
public void run() {
try {
appLogAggregator.run();
} finally {
* appLogAggregators.remove(appId);
closeFileSystems(userUgi);*
}
}
};
{noformat}
If the aggregatorWrapper runnable runs before the enableAppAggregators is
called for that app during credential updates, then enableAppAggregators will
not find the app in appLogAggregators map since in finally
appLogAggregators.remove(appId) is called ;i.e
{noformat}
AppLogAggregator aggregator = appLogAggregators.get(appId);
if (aggregator != null) {
{noformat}
Am I missing something here?
2. Applications dont get removed from invalidTokenApps except in
enableAppAggregators. This cause a leak if we never get a credential update
event for some reason? Should we add logic to remove when application finishes
i.e APPLICATION_FINISHED is received?
3. Minor nit - In ContainerManagerImp.handleCredentialUpdate, can we rename
disableApps to disabledApps
{noformat}
Set<ApplicationId> disableApps = logHandler.getDisabledApps();
{noformat}
> App local logs could leaked if log aggregation fails to initialize for the app
> ------------------------------------------------------------------------------
>
> Key: YARN-8418
> URL: https://issues.apache.org/jira/browse/YARN-8418
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.8.0, 3.0.0-alpha1
> Reporter: Bibin A Chundatt
> Assignee: Bibin A Chundatt
> Priority: Critical
> Attachments: YARN-8418.001.patch, YARN-8418.002.patch,
> YARN-8418.003.patch, YARN-8418.004.patch, YARN-8418.005.patch
>
>
> If log aggregation fails init createApp directory container logs could get
> leaked in NM directory
> For log running application restart of NM after token renewal this case is
> possible/ Application submission with invalid delegation token
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
