[
https://issues.apache.org/jira/browse/YARN-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15942651#comment-15942651
]
zhihai xu commented on YARN-6396:
---------------------------------
I attached a patch which call verifyAndCreateRemoteLogDir in serviceStart, only
call verifyAndCreateRemoteLogDir in initApp, when remote log directory is
failed to create.
> Call verifyAndCreateRemoteLogDir at service initialization instead of
> application initialization to decrease load for name node
> -------------------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-6396
> URL: https://issues.apache.org/jira/browse/YARN-6396
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: log-aggregation
> Affects Versions: 3.0.0-alpha2
> Reporter: zhihai xu
> Assignee: zhihai xu
> Priority: Minor
> Attachments: YARN-6396.000.patch
>
>
> Call verifyAndCreateRemoteLogDir at service initialization instead of
> application initialization to decrease load for name node.
> Currently for every application at each Node, verifyAndCreateRemoteLogDir
> will be called before doing log aggregation, This will be a non trivial
> overhead for name node in a large cluster since verifyAndCreateRemoteLogDir
> calls getFileStatus. Once the remote log directory is created successfully,
> it is not necessary to call it again. It will be better to call
> verifyAndCreateRemoteLogDir at LogAggregationService service initialization.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
