Wangda Tan commented on YARN-3139:

Thanks [~jianhe] for review:

bq. Is removing these locks ok?..

The original lock is added by YARN-4519, but I found instead of locking on leaf 
queue, we can simply lock the application. When an application is locked, we 
cannot release/increase/decrease a container while another 
release/increase/decrease container is in progress. So we will be safe if all 
these operations are protected by application's writeLock.

And here is a summary of how scheduler/queue/app locks being used after this 
rollback/decreaseContainer/completedContainer: LeafQueue.writeLock -> 
updateRequest/updateIncreaseRequest : app.writeLock
new allocation/new reservation/remove or add app/remove or add 
node/reinitialize : CS.writeLock -> PQ.writeLock -> LQ.writeLock -> 

So I think existing model looks fine.

bq. Also, here.. the queue lock is removed
Same as above

bq. In fairscheduler, here, locking on application is changed to locking on 
scheduler ?

bq. the lock is outside try block...
Actually this is a workaround for the previously reported findbugs warning: 
This is safe and it is suggested, because when writeLock.lock() fails (like 
throw any exception), the writeLock.unlock() will not be invoked. However in 
practice, putting writelock inside try or out of try are both safe, because 
writeLock.lock() will not likely throw any exception.

Uploading ver.2 patch, fixed a couple of unnecessary readlocks.

> Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler
> ----------------------------------------------------------------------
>                 Key: YARN-3139
>                 URL: https://issues.apache.org/jira/browse/YARN-3139
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager, scheduler
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-3139.0.patch, YARN-3139.1.patch, YARN-3139.2.patch
> Enhance locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler, as 
> mentioned in YARN-3091, a possible solution is using read/write lock. Other 
> fine-graind locks for specific purposes / bugs should be addressed in 
> separated tickets.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to