[
https://issues.apache.org/jira/browse/YARN-6521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985230#comment-15985230
]
Robert Kanter commented on YARN-6521:
-------------------------------------
I think what [~piaoyu zhang] is asking is that when you disable log aggregation
(because it was making too many small files and causing a problem for the NN),
he still wants to be able to get logs from the YARN CLI logs command, which
requires log aggregation.
[~piaoyu zhang], instead of disabling log aggregation, you could periodically
run the Archive Logs tool
(https://hadoop.apache.org/docs/r2.8.0/hadoop-archive-logs/HadoopArchiveLogs.html),
which combines all of the log files for an Application into one file. The
YARN CLI logs command and JHS can still read these files. It does require
Hadoop 2.8.0+ though.
> Yarn Log-aggregation Transform Enable not to Spam the NameNode
> --------------------------------------------------------------
>
> Key: YARN-6521
> URL: https://issues.apache.org/jira/browse/YARN-6521
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: zhangyubiao
>
> Nowdays We have large cluster,with the apps incr and container incr ,We split
> an namespace to store applogs.
> But the log-aggregation /tmp/app-logs incr also make the namespace responce
> slow.
> We want to chang yarn.log-aggregation-enable true -> false,But Transform the
> the yarn log cli service also can get the app-logs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
