Jason Lowe reassigned YARN-4047:

    Assignee: Jason Lowe

In OOZIE-1729 Oozie started calling getApplications to look for applications 
with specific tags.  This significantly increases the utilization of this 
method on a cluster that makes heavy use of Oozie.

One quick fix for the Oozie use-case may be to swap the filter order.  Rather 
than doing the expensive checkAccess call first, we can do all the other 
filtering first and finally verify the user has access before adding the app to 
the response.  In the Oozie scenario most apps will be filtered by the tag 
check before we ever get to the checkAccess call.

> ClientRMService getApplications has high scheduler lock contention
> ------------------------------------------------------------------
>                 Key: YARN-4047
>                 URL: https://issues.apache.org/jira/browse/YARN-4047
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
> The getApplications call can be particuarly expensive because the code can 
> call checkAccess on every application being tracked by the RM.  checkAccess 
> will often call scheduler.checkAccess which will grab the big scheduler lock. 
>  This can cause a lot of contention with the scheduler thread which is busy 
> trying to process node heartbeats, app allocation requests, etc.

This message was sent by Atlassian JIRA

Reply via email to