Haibo Chen created YARN-4766:
--------------------------------
Summary: NM should not aggregate logs older than the retention
policy
Key: YARN-4766
URL: https://issues.apache.org/jira/browse/YARN-4766
Project: Hadoop YARN
Issue Type: Bug
Components: log-aggregation, nodemanager
Reporter: Haibo Chen
Assignee: Haibo Chen
When a log aggregation fails on the NM the information is for the attempt is
kept in the recovery DB. Log aggregation can fail for multiple reasons which
are often related to HDFS space or permissions.
On restart the recovery DB is read and if an application attempt needs its logs
aggregated, the files are scheduled for aggregation without any checks. The log
files could be older than the retention limit in which case we should not
aggregate them but immediately mark them for deletion from the local file
system.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
