[
https://issues.apache.org/jira/browse/YARN-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15968642#comment-15968642
]
Jian He commented on YARN-6396:
-------------------------------
Hi [~zxu],
only one thing is that if the valid permission becomes invalid maybe by
accident at some point, the verifyAndCreateRemoteLogDir won't be invoked any
more to validate the permission, the log-aggregation will eventually fail.
I think it's a trade off between validation and efficiency.
I'm ok with current approach.
[~xgong], do you have comments ?
> Call verifyAndCreateRemoteLogDir at service initialization instead of
> application initialization to decrease load for name node
> -------------------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-6396
> URL: https://issues.apache.org/jira/browse/YARN-6396
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: log-aggregation
> Affects Versions: 3.0.0-alpha2
> Reporter: zhihai xu
> Assignee: zhihai xu
> Priority: Minor
> Attachments: YARN-6396.000.patch
>
>
> Call verifyAndCreateRemoteLogDir at service initialization instead of
> application initialization to decrease load for name node.
> Currently for every application at each Node, verifyAndCreateRemoteLogDir
> will be called before doing log aggregation, This will be a non trivial
> overhead for name node in a large cluster since verifyAndCreateRemoteLogDir
> calls getFileStatus. Once the remote log directory is created successfully,
> it is not necessary to call it again. It will be better to call
> verifyAndCreateRemoteLogDir at LogAggregationService service initialization.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
