[
https://issues.apache.org/jira/browse/YARN-4773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15185757#comment-15185757
]
Jason Lowe commented on YARN-4773:
----------------------------------
It is not fixed by YARN-4720 since this is a listStatus call not a
getFileStatus call. Specifically the call is coming from
AppLogAggregatorImpl#cleanOldLogs where it does a listStatus looking for old
logs to remove. If rolling log aggregation is disabled then the listStatus
call per node is unnecessary. We would only be uploading logs when the
application completes, and there should be no old logs to remove in that
scenario.
> Log aggregation performs extraneous filesystem operations when rolling log
> aggregation is disabled
> --------------------------------------------------------------------------------------------------
>
> Key: YARN-4773
> URL: https://issues.apache.org/jira/browse/YARN-4773
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 2.6.0
> Reporter: Jason Lowe
> Priority: Minor
>
> I noticed when log aggregation occurs for an application the nodemanager is
> listing the application's log directory in HDFS. Apparently this is for
> removing old logs before uploading new ones. This is a wasteful operation
> when rolling log aggregation is disabled, since there will be no prior logs
> in HDFS -- aggregation only occurs once when rolling log aggregation is
> disabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
