[
https://issues.apache.org/jira/browse/YARN-3137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16006467#comment-16006467
]
Jason Lowe commented on YARN-3137:
----------------------------------
Seeing this lock contention quite a bit in 2.8, and I guess I'm still a little
confused why we need a full CapacityScheduler lock to do this. Even if we have
a lock, we're either checking just before or just after some kind of hierarchy
change. Queue lookup is not by path but simply a concurrent hashmap lookup, so
that's not an issue. Either the queue is there or it's not. Then we can
invoke checkAccess on the queue itself. The queue knows its path in the
hierarchy, and again either we're checking before the queue is moved or after.
As long as we lock the specific queue when trying to check the ACL we should be
fine, but maybe I'm missing something.
Can someone elaborate the scenario where moving the checkAccess lock from the
highly-contended CapacityScheduler lock to a queue-specific lock causes
problems when the hierarchy changes?
> CapacityScheduler.checkAccess unnecessarily grabs the scheduler lock
> --------------------------------------------------------------------
>
> Key: YARN-3137
> URL: https://issues.apache.org/jira/browse/YARN-3137
> Project: Hadoop YARN
> Issue Type: Sub-task
> Affects Versions: 2.5.0
> Reporter: Jason Lowe
> Assignee: Varun Saxena
>
> The queues are stored in a ConcurrentHashMap and the code already checks for
> a null queue. At best we need to lock individual queues when processing the
> access check, but I don't see why we need to grab the highly-contested
> scheduler lock to lookup which queue to use for the hasAccess call.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
