[
https://issues.apache.org/jira/browse/YARN-4325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15279212#comment-15279212
]
Junping Du commented on YARN-4325:
----------------------------------
Thanks [~jlowe] for comments. I agree that adding a boolean flag may not be a
very decent solution. However, given APPLICATION_LOG_HANDLING_FAILED event
could be sent when app is still running, we have to cache this event (in
boolean flag or some other way) so later we can do app cleanup when app is
finished.
Do we have better way to achieve this?
> Purge app state from NM state-store should cover more LOG_HANDLING cases
> ------------------------------------------------------------------------
>
> Key: YARN-4325
> URL: https://issues.apache.org/jira/browse/YARN-4325
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.6.0
> Reporter: Junping Du
> Assignee: Junping Du
> Priority: Critical
> Attachments: ApplicationImpl.PNG, YARN-4325-v1.1.patch,
> YARN-4325-v1.patch, YARN-4325.patch
>
>
> From a long running cluster, we found tens of thousands of stale apps still
> be recovered in NM restart recovery.
> After investigating, there are three issues cause app state leak in NM
> state-store:
> 1. APPLICATION_LOG_HANDLING_FAILED is not handled with remove App in
> NMStateStore.
> 2. APPLICATION_LOG_HANDLING_FAILED event is missing in sent when hit
> aggregator's doAppLogAggregation() exception case.
> 3. Only Application in FINISHED status receiving APPLICATION_LOG_FINISHED has
> transition to remove app in NM state store. Application in other status -
> like APPLICATION_RESOURCES_CLEANUP will ignore the event and later forget to
> remove this app from NM state store even after app get finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
