[
https://issues.apache.org/jira/browse/YARN-194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13704115#comment-13704115
]
Bikas Saha commented on YARN-194:
---------------------------------
Why does the NM upload logs when it the container completes? It does not need
to wait for app completion. It can use HDFS append to append the logs to the
same file. This is safe since NM should be the single writer.
NM could then delete these container logs after uploading them. Risk is
duplicate data whenever NM restarts while it was in the middle of uploading a
particular log.
> Log handling in case of NM restart.
> -----------------------------------
>
> Key: YARN-194
> URL: https://issues.apache.org/jira/browse/YARN-194
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 0.23.4
> Reporter: Siddharth Seth
> Assignee: Omkar Vinit Joshi
>
> Currently, if an NM restarts - existing logs will be left around till they're
> manually cleaned up. The NM could be improved to handle these files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
