[
https://issues.apache.org/jira/browse/YARN-5773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15601974#comment-15601974
]
Varun Saxena commented on YARN-5773:
------------------------------------
Thanks [~bibinchundatt] for filing the JIRA.
Agree that we do not need to iterate over all the pending apps on recovery as
NMs' are not yet registered.
If there are large number of running apps, RM unnecessarily spends quite a bit
of time in this loop.
Applications can be activated as and when nodes are added.
> Skip LeafQueue#activateApplication for running application on recovery
> ----------------------------------------------------------------------
>
> Key: YARN-5773
> URL: https://issues.apache.org/jira/browse/YARN-5773
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Bibin A Chundatt
> Assignee: Bibin A Chundatt
> Priority: Critical
>
> # Submit application 10K application to default queue.
> # All applications are in accepted state
> # Now restart resourcemanager
> For each application recovery {{LeafQueue#activateApplications()}} is
> invoked.Resulting in AM limit check to be done even before Node managers are
> getting registered.
> Total iteration for N application is about {{N(N+1)/2}} for {{10K}}
> application {{50000000}} iterations causing time take for Rm to be active
> more than 10 min.
> Since NM resources are not yet added to during recovery we should skip
> {{activateApplicaiton()}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
