[
https://issues.apache.org/jira/browse/YARN-5237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327744#comment-15327744
]
Junping Du commented on YARN-5237:
----------------------------------
Hi [~xgong], thanks for reporting the issue and delivering the fix. I can
understand the problem here, however, I am concerned that if we get rid of
{{containerLogAggregators.remove(container);}} when containers get finished, it
could end up with too many ContainerLogAggregators threads in case an app (long
running service) allocate a lot of containers. Can we think some other way to
address the problem? Something like: we can upload the container log to final
location of HDFS in case of rolling log aggregation case (include pattern)?
> Not all logs get aggregated with rolling log aggregation.
> ---------------------------------------------------------
>
> Key: YARN-5237
> URL: https://issues.apache.org/jira/browse/YARN-5237
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Xuan Gong
> Assignee: Xuan Gong
> Attachments: YARN-5237.1.patch, YARN-5237.2.patch, YARN-5237.3.patch
>
>
> Steps to reproduce:
> 1) enable RM recovery
> 2) Run a sleep job
> 3) restart RM
> 4) kill the application
> We can not find that the logs for the first attempt
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
