[ 
https://issues.apache.org/jira/browse/YARN-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387114#comment-14387114
 ] 

Sunil G commented on YARN-3136:
-------------------------------

Hi [~jlowe] [~jianhe]
{noformat}
Bug type IS2_INCONSISTENT_SYNC (click for details) 
In class 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler
Field 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler.applications
Synchronized 90% of the time
Unsynchronized access at AbstractYarnScheduler.java:[line 138]
Unsynchronized access at AbstractYarnScheduler.java:[line 165]
Unsynchronized access at AbstractYarnScheduler.java:[line 233]
{noformat}

As "applications" is now a concurrent version, I feel we do not need a lock. 
Kindly share your opinion.

test case failure is not related.



> getTransferredContainers can be a bottleneck during AM registration
> -------------------------------------------------------------------
>
>                 Key: YARN-3136
>                 URL: https://issues.apache.org/jira/browse/YARN-3136
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: scheduler
>    Affects Versions: 2.6.0
>            Reporter: Jason Lowe
>            Assignee: Sunil G
>         Attachments: 0001-YARN-3136.patch, 0002-YARN-3136.patch, 
> 0003-YARN-3136.patch, 0004-YARN-3136.patch, 0005-YARN-3136.patch, 
> 0006-YARN-3136.patch, 0007-YARN-3136.patch, 0008-YARN-3136.patch, 
> 0009-YARN-3136.patch
>
>
> While examining RM stack traces on a busy cluster I noticed a pattern of AMs 
> stuck waiting for the scheduler lock trying to call getTransferredContainers. 
>  The scheduler lock is highly contended, especially on a large cluster with 
> many nodes heartbeating, and it would be nice if we could find a way to 
> eliminate the need to grab this lock during this call.  We've already done 
> similar work during AM allocate calls to make sure they don't needlessly grab 
> the scheduler lock, and it would be good to do so here as well, if possible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to