[
https://issues.apache.org/jira/browse/YARN-9634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16869268#comment-16869268
]
Weiwei Yang commented on YARN-9634:
-----------------------------------
Hi [~zhuqi]
Thanks. Are you suggesting to let parameter
\{{yarn.nodemanager.remote-app-log-dir}} support multiple dirs, and use e.g
round-robin policy to select dirs? If this is the case, will this break
something? Such as history server, timeline-server or log CLI etc, I assume
some of these components need to read logs from this location. Please
elaborate, thanks.
> Make yarn submit dir and log aggregation dir more evenly distributed
> --------------------------------------------------------------------
>
> Key: YARN-9634
> URL: https://issues.apache.org/jira/browse/YARN-9634
> Project: Hadoop YARN
> Issue Type: Improvement
> Affects Versions: 3.2.0
> Reporter: zhuqi
> Assignee: zhuqi
> Priority: Major
>
> When the cluster size is large, the dir which user submits the job, and the
> dir which container log aggregate, and other information will fill the HDFS
> directory, because the HDFS directory has a default storage limit. In
> response to this situation, we can change these dirs more distributed, with
> some policy to choose, such as hash policy and round robin policy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
