[
https://issues.apache.org/jira/browse/YARN-549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13631990#comment-13631990
]
Zhijie Shen commented on YARN-549:
----------------------------------
After application store is moved out of ClientRMService#submitApplicaton, the
remaining operations in this function are:
1. Checking whether the submitted application is unique
2. Checking whether the requested AM resource is valid
3. Calling RMAppManager to handle the app submission event directly instead of
scheduling it through AsyncDispatcher
On the other side, RMAppManager#submitApplcation, which is ultimately called by
ClientRMService#submitApplicaton contain the following operations:
1. Checking whether the queue of ASC is null
2. Checking whether the application name is null
3. Creating the RMApp instance
4. Checking whether the submitted application is unique
5. Adding the RMApp instance to ApplicationACLsManager
6. Adding the RMApp instance to DelegationTokenRenewer
7. Kicking the RMApp instance to the state machine to start (or to be rejected
if some exception is caught)
IMHO, application submission can refactored with the following rules:
1. All the aforementioned operations are not slow, such that they don't need to
be delayed.
2. It is confusing that part of sanity check is RMAppManager#submitApplcation
while the other is in ClientRMService#submitApplicaton. 1 and 2 of
ClientRMService#submitApplicaton should be moved into
RMAppManager#submitApplcation. In addition, 4 of RMAppManager#submitApplcation
doesn't need to be done twice.
3. Instead of directly calling RMAppManager to handle the app submission event,
ClientRMService#submitApplicaton should call RMAppManager#submitApplcation
directly, because the existing code is a fake event scheduling method, and is
somewhat misleading. Calling RMAppManager#submitApplcation directly has the
equivalent results. Then, in ClientRMService#submitApplicaton, there's some
simple code to call RMAppManager#submitApplcation and do the log.
4. RMAppManager#submitApplcation adds the "synchronized" modifier in order to
prevent interleaving the operations in RMAppManager#submitApplcation if two
clients submitted applications simultaneously.
5. RMAppManager#submitApplcation throws YarnRemoteException, which will finally
be passed to the client if the submission has some exception, as what is done
currently.
AFAIK, the aforementioned changes will not need the changes at the client side.
> Asynchronous application submission
> -----------------------------------
>
> Key: YARN-549
> URL: https://issues.apache.org/jira/browse/YARN-549
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Zhijie Shen
> Assignee: Zhijie Shen
> Attachments: Proposal of Asynchronous Application Submission_v1.pdf
>
>
> Currently, when submitting an application, storeApplication will be called
> for recovery. However, it is a blocking API, and is likely to block
> concurrent application submissions. Therefore, it is good to make application
> submission asynchronous, and postpone storeApplication.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
