Sunil G commented on YARN-3136:

1. As you mentioned a full lock on Scheduler is grabbed for getting 
liveContainers (except am container) from FicaSchedulerApp.  This comes from 
below map.

protected Map<ApplicationId, SchedulerApplication<T>> applications;

And I could see that, most of the methods are grabbing a Scheduler lock  while 
accessing this map.

2. *rmContext.getRMApps()*. Yes, this is a concurrent map and i feel this need 
not to be surrounded by a lock in scheduler level.

I feel, we could improve getTransferredContainers like below:
A getter can be used under scheduler lock to get the FicaSchedulerApp object 
within *getTransferredContainers* . Also we can remove the scheduler lock from 
getTransferredContainers. This must be safe enough. 

[~varun_saxena] As i have done some ground work, is it ok if i assign this 
ticket to myself :). 
[~jlowe] pls share your thoughts on this approach. Also I would like to take 
this jira, pls feel free to takeover otherwise :).

> getTransferredContainers can be a bottleneck during AM registration
> -------------------------------------------------------------------
>                 Key: YARN-3136
>                 URL: https://issues.apache.org/jira/browse/YARN-3136
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: scheduler
>    Affects Versions: 2.6.0
>            Reporter: Jason Lowe
>            Assignee: Varun Saxena
> While examining RM stack traces on a busy cluster I noticed a pattern of AMs 
> stuck waiting for the scheduler lock trying to call getTransferredContainers. 
>  The scheduler lock is highly contended, especially on a large cluster with 
> many nodes heartbeating, and it would be nice if we could find a way to 
> eliminate the need to grab this lock during this call.  We've already done 
> similar work during AM allocate calls to make sure they don't needlessly grab 
> the scheduler lock, and it would be good to do so here as well, if possible.

This message was sent by Atlassian JIRA

Reply via email to