Jason Lowe commented on YARN-2176:

That proposal would work for the deactivate path, but how does it work for the 
activate case?  If the queue is not normally looping over the deactivated apps 
then it is not going to call hasPendingRequests() on them and we won't ever add 
it back to the list of active apps to iterate.  If we do always loop over the 
deactivated apps to call this then that sorta defeats a large portion of the 
optimization.  There needs to be more than a predicate function for the leaf 
queue to call, unless I'm missing something.

> CapacityScheduler loops over all running applications rather than actively 
> requesting apps
> ------------------------------------------------------------------------------------------
>                 Key: YARN-2176
>                 URL: https://issues.apache.org/jira/browse/YARN-2176
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler
>    Affects Versions: 2.4.0
>            Reporter: Jason Lowe
> The capacity scheduler performance is primarily dominated by 
> LeafQueue.assignContainers, and that currently loops over all applications 
> that are running in the queue.  It would be more efficient if we looped over 
> just the applications that are actively asking for resources rather than all 
> applications, as there could be thousands of applications running but only a 
> few hundred that are currently asking for resources.

This message was sent by Atlassian JIRA

Reply via email to