[
https://issues.apache.org/jira/browse/YARN-4771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated YARN-4771:
------------------------------------------
Target Version/s: 2.7.4 (was: 2.7.3)
bq. However it may have issues with very long-running apps that churn a lot of
containers, since the container state won't be released until the application
completes.
This is going to be problematic, impacting NM memory usage.
I think the right solution is to decouple log-aggregation state completely from
the rest of the container-state, and persist that separately in state-store etc
irrespective of container / application state.
IAC, I am dropping this off 2.7.3 given that this is a long standing issue and
as I'd to proceed with that release now.
> Some containers can be skipped during log aggregation after NM restart
> ----------------------------------------------------------------------
>
> Key: YARN-4771
> URL: https://issues.apache.org/jira/browse/YARN-4771
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 2.7.2
> Reporter: Jason Lowe
> Priority: Critical
> Attachments: YARN-4771.001.patch, YARN-4771.002.patch
>
>
> A container can be skipped during log aggregation after a work-preserving
> nodemanager restart if the following events occur:
> # Container completes more than
> yarn.nodemanager.duration-to-track-stopped-containers milliseconds before the
> restart
> # At least one other container completes after the above container and before
> the restart
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
