[
https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057274#comment-17057274
]
Wangda Tan edited comment on YARN-9879 at 3/11/20, 5:55 PM:
------------------------------------------------------------
Thanks [~shuzirra] for uploading another monster patch!
I didn't check other places, I only checked the CSQueueStore related items:
*Nits: *
- CapacitySchedulerQueueManager#getShortNameQueues please mark @VisibleByTests
- Similarily, mark CSQueueStore#getShortNameQueues @VisibleByTests
*Primary: Locking of the class still have many issues: *
For all methods will be accessed by external class. Make sure that:
1) Avoid using synchronized lock when read/write lock present.
2) ALL external read-only methods protected by readlock.
3) ALL external writable methods protected by writelock.
4) Use
{code}
try {
lock.(read/or write).lock()
.. your logic ..
} catch (exception) {
// if there's any
} finally {
lock.(read/or write).unlock()
}
{code}
To make sure lock is always released: Example: CapacityScheduler#serviceStop
5) After the above changes, you can remove all usage of {{ConcurrentHashMap}},
it is bad for performance with locks. Hashmap will be way faster under the
protection of lock.
was (Author: wangda):
Thanks [~shuzirra] for uploading another monster patch!
I didn't check other places, I only checked the CSQueueStore related items:
*Nits: *
- CapacitySchedulerQueueManager#getShortNameQueues please mark @VisibleByTests
- Similarily, mark CSQueueStore#getShortNameQueues @VisibleByTests
*Primary: Locking of the class still have many issues: *
For all methods will be accessed by external class. Make sure that:
1) There're no synchronized lock.
2) All external read-only method use readlock.
3) All external writable method use writelock.
4) Use
{code}
try {
lock.(read/or write).lock()
.. your logic ..
} catch (exception) {
// if there's any
} finally {
lock.(read/or write).unlock()
}
{code}
To make sure lock is always released: Example: CapacityScheduler#serviceStop
5) After above changes, you can remove all usage of {{ConcurrentHashMap}}, it
is bad for performance with locks. Hashmap will be way faster under protection
of lock.
> Allow multiple leaf queues with the same name in CS
> ---------------------------------------------------
>
> Key: YARN-9879
> URL: https://issues.apache.org/jira/browse/YARN-9879
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Gergely Pollak
> Assignee: Gergely Pollak
> Priority: Major
> Labels: fs2cs
> Attachments: CSQueue.getQueueUsage.txt, DesignDoc_v1.pdf,
> YARN-9879.POC001.patch, YARN-9879.POC002.patch, YARN-9879.POC003.patch,
> YARN-9879.POC004.patch, YARN-9879.POC005.patch, YARN-9879.POC006.patch,
> YARN-9879.POC007.patch, YARN-9879.POC008.patch, YARN-9879.POC009.patch,
> YARN-9879.POC010.patch, YARN-9879.POC011.patch, YARN-9879.POC012.patch
>
>
> Currently the leaf queue's name must be unique regardless of its position in
> the queue hierarchy.
> Design doc and first proposal is being made, I'll attach it as soon as it's
> done.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
