Rohith Sharma K S created YARN-4755:
---------------------------------------
Summary: Optimize sending appACLsUpdated event to TimelineServer
while recovering completed applications
Key: YARN-4755
URL: https://issues.apache.org/jira/browse/YARN-4755
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Rohith Sharma K S
In method {{RMAppManager#createAndPopulateNewRMApp}}, appACLsUpdated is sent to
timelineserver for every application that get created.
{code}
private RMAppImpl createAndPopulateNewRMApp(
ApplicationSubmissionContext submissionContext, long submitTime,
String user, boolean isRecovery) throws YarnException {
//
//
String appViewACLs = submissionContext.getAMContainerSpec()
.getApplicationACLs().get(ApplicationAccessType.VIEW_APP);
rmContext.getSystemMetricsPublisher().appACLsUpdated(
application, appViewACLs, System.currentTimeMillis());
return application;
}
{code}
Say if we have 10K completed applications to recover, 30K events will be
generated i.e app_created, app_finished and app_acl_updated. For completed
applications, I think need not to send app-acl-updated event with which
gradually reduce load on the dispatcher.
Eventhough MultiDispatcher is used to publish timeline events, it is bottle
neck when max-completed is configured very high value may be 100K.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
