[
https://issues.apache.org/jira/browse/YARN-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499232#comment-13499232
]
Siddharth Seth commented on YARN-162:
-------------------------------------
The patch does change behaviour. Prior to this, directory creation failure
would cause the app to be marked as FAILED on the NM. Failing to aggregate logs
for some containers would not fail the app though. With the patch, dir creation
failing does not fail the app.
> nodemanager log aggregation has scaling issues with namenode
> ------------------------------------------------------------
>
> Key: YARN-162
> URL: https://issues.apache.org/jira/browse/YARN-162
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 0.23.3
> Reporter: Nathan Roberts
> Assignee: Vinod Kumar Vavilapalli
> Priority: Critical
> Attachments: YARN-162.txt, YARN-162_WIP.txt
>
>
> Log aggregation causes fd explosion on the namenode. On large clusters this
> can exhaust FDs to the point where datanodes can't check-in.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
