[
https://issues.apache.org/jira/browse/YARN-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15969199#comment-15969199
]
Haibo Chen commented on YARN-6396:
----------------------------------
Robert and Jian's comments remind me of that we have seen customers delete the
log directory every once in a while to clean up, which I think is a fair thing
to do. Maybe we want to make it configurable (true by default) whether we want
to call verifyAndCreateRemoteLogDir at LogAggregationService service
initialization?
> Call verifyAndCreateRemoteLogDir at service initialization instead of
> application initialization to decrease load for name node
> -------------------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-6396
> URL: https://issues.apache.org/jira/browse/YARN-6396
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: log-aggregation
> Affects Versions: 3.0.0-alpha2
> Reporter: zhihai xu
> Assignee: zhihai xu
> Priority: Minor
> Attachments: YARN-6396.000.patch
>
>
> Call verifyAndCreateRemoteLogDir at service initialization instead of
> application initialization to decrease load for name node.
> Currently for every application at each Node, verifyAndCreateRemoteLogDir
> will be called before doing log aggregation, This will be a non trivial
> overhead for name node in a large cluster since verifyAndCreateRemoteLogDir
> calls getFileStatus. Once the remote log directory is created successfully,
> it is not necessary to call it again. It will be better to call
> verifyAndCreateRemoteLogDir at LogAggregationService service initialization.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
