[
https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17036663#comment-17036663
]
Gergely Pollak commented on YARN-9879:
--------------------------------------
In this latest patch I've checked all CSQueue.getQueueName calls, and replaced
them with getQueuePath, also checked how those values were used, and changed
the code if it was necessary. The biggest change was around the placement
rules, but I think I've managed to solve the issues with minimal impact and
code changes.
The problem with placement rules is they can dynamically create colliding queue
names, and at configuration time it is impossible to determine whether the rule
will collide with an already existing queue or not. So in this implementation,
I changed the config parsing methods to be aware of the the possible name
collisions and also added an extra ambiguity check just before creating the
placement context, this way the user will be able to see in the logs why the
application fails.
The expected behaviour for name collisions is to refuse the application
submission if the matching placement rule results in an ambiguous queue name.
This is consistent with the app submission behaviour when providing a queue
name, but it is a leaf name only.
Next step is to check the application submission side, and make sure the
submitted queue names are parsed properly and are compatible with the internal
full queue name usage.
Also marked a few parts of the code for cleanup later, we are using 3 classes
with very similar behaviour and it was really confusing to understand the
difference between them (practically there is none), but since this patch is
already growing in size, and I haven't even touched the tests, I'll file
separate Jiras about those later.
> Allow multiple leaf queues with the same name in CS
> ---------------------------------------------------
>
> Key: YARN-9879
> URL: https://issues.apache.org/jira/browse/YARN-9879
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Gergely Pollak
> Assignee: Gergely Pollak
> Priority: Major
> Labels: fs2cs
> Attachments: CSQueue.getQueueUsage.txt, DesignDoc_v1.pdf,
> YARN-9879.POC001.patch, YARN-9879.POC002.patch, YARN-9879.POC003.patch
>
>
> Currently the leaf queue's name must be unique regardless of its position in
> the queue hierarchy.
> Design doc and first proposal is being made, I'll attach it as soon as it's
> done.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
