[
https://issues.apache.org/jira/browse/YARN-4766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haibo Chen updated YARN-4766:
-----------------------------
Attachment: yarn4766.006.patch
Rebase the patch on trunk, addressed some of the checkstyle issues. Many of
them are related to accessibility of the fields. The fields are all final and
the class to which the fields belong is private, so IMHO, the warnings are not
necessarily valid. The javadoc issue is on a protobuf-generated file.
> NM should not aggregate logs older than the retention policy
> ------------------------------------------------------------
>
> Key: YARN-4766
> URL: https://issues.apache.org/jira/browse/YARN-4766
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: log-aggregation, nodemanager
> Reporter: Haibo Chen
> Assignee: Haibo Chen
> Attachments: yarn4766.001.patch, yarn4766.002.patch,
> yarn4766.003.patch, yarn4766.004.patch, yarn4766.004.patch,
> yarn4766.005.patch, yarn4766.006.patch
>
>
> When a log aggregation fails on the NM the information is for the attempt is
> kept in the recovery DB. Log aggregation can fail for multiple reasons which
> are often related to HDFS space or permissions.
> On restart the recovery DB is read and if an application attempt needs its
> logs aggregated, the files are scheduled for aggregation without any checks.
> The log files could be older than the retention limit in which case we should
> not aggregate them but immediately mark them for deletion from the local file
> system.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
