[
https://issues.apache.org/jira/browse/YARN-4755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15175693#comment-15175693
]
Naganarasimha G R commented on YARN-4755:
-----------------------------------------
I think we discussed these topics in the following JIRAs' YARN-3127 and
YARN-4392 and the conclusion was, we were ok with republishing the events with
exact data rather than not publishing at all because its not guaranteed that
ATS events for apps in state store are successfully published.
To actually see *appACLsUpdated* need not be separately published again we can
directly publish this operation along with appCreatedEvent thus avoids one
additional entity processing. Need to check what would be the ideal place to
have acl information and also ensure its compatible with the current code.
> Optimize sending appACLsUpdated event to TimelineServer while recovering
> completed applications
> -----------------------------------------------------------------------------------------------
>
> Key: YARN-4755
> URL: https://issues.apache.org/jira/browse/YARN-4755
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Rohith Sharma K S
> Assignee: Naganarasimha G R
>
> In method {{RMAppManager#createAndPopulateNewRMApp}}, appACLsUpdated is sent
> to timelineserver for every application that get created.
> {code}
> private RMAppImpl createAndPopulateNewRMApp(
> ApplicationSubmissionContext submissionContext, long submitTime,
> String user, boolean isRecovery) throws YarnException {
> //
> //
> String appViewACLs = submissionContext.getAMContainerSpec()
> .getApplicationACLs().get(ApplicationAccessType.VIEW_APP);
> rmContext.getSystemMetricsPublisher().appACLsUpdated(
> application, appViewACLs, System.currentTimeMillis());
> return application;
> }
> {code}
> Say if we have 10K completed applications to recover, 30K events will be
> generated i.e app_created, app_finished and app_acl_updated. For completed
> applications, I think need not to send app-acl-updated event with which
> gradually reduce load on the dispatcher.
> Eventhough MultiDispatcher is used to publish timeline events, it is bottle
> neck when max-completed is configured very high value may be 100K.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
