[
https://issues.apache.org/jira/browse/YARN-4766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200655#comment-15200655
]
Haibo Chen commented on YARN-4766:
----------------------------------
fixed the liscense + checkstyle issues. The unit test failure is unrelated to
the patch. I have create another jira to fix the test failure at
https://issues.apache.org/jira/browse/YARN-4838
> NM should not aggregate logs older than the retention policy
> ------------------------------------------------------------
>
> Key: YARN-4766
> URL: https://issues.apache.org/jira/browse/YARN-4766
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: log-aggregation, nodemanager
> Reporter: Haibo Chen
> Assignee: Haibo Chen
> Attachments: yarn4766.001.patch, yarn4766.002.patch
>
>
> When a log aggregation fails on the NM the information is for the attempt is
> kept in the recovery DB. Log aggregation can fail for multiple reasons which
> are often related to HDFS space or permissions.
> On restart the recovery DB is read and if an application attempt needs its
> logs aggregated, the files are scheduled for aggregation without any checks.
> The log files could be older than the retention limit in which case we should
> not aggregate them but immediately mark them for deletion from the local file
> system.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
