[
https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010190#comment-17010190
]
Wilfred Spiegelenburg commented on YARN-9879:
---------------------------------------------
Thank you [~leftnoteasy] for the comments.
{quote}And once application is submitted to CS, internal to CS, we should make
sure we use queue path instead of queue name at all other places. Otherwise we
will complicate other logics.
{quote}
I agree that is what I had in mind too. Make it as simple as possible inside
the scheduler and that is to use just the full path internally.
For the configuration change: I do not think it is a problem and we can just
accept the change. To be fair to the administrator we should show a message
when the configuration is loaded or changed and the leaf queues are not unique
(any more). However that is probably as far as we need to go.
{quote}Instead of using scheduler.getQueue, we may need to consider to add a
method like getAppSubmissionQueue() to get a queue based on path or name, and
after that, we will put normalized queue_path back to submission context of
application to make sure in the future inside scheduler we all refer to queue
path.
{quote}
The FS already does something like this already because it uses a placement
rule in all cases. We should leverage a similar mechanism in the CS. We pass
the queue from the submission into the queue placement which handles the full
path or not. In both cases it just passes back the queue object which will be
using the full path. If the queue is not found or the queue name is not unique
it fails as per normal. The returned queue info is updated in the app and
submission context.
Far simpler than putting the burden on the core scheduler. It is all hidden in
the placement of the app into the queue using the placement engine.
I did not mention queue mapping in my design. Queue mapping itself I thought
did not need to change. We already calculate the parent queue in the rules if I
am correct so the only change would be the return value. We do all internal
handling for queues with the full queue path so it is a logical change. Using
the placement rule for the qualified or not qualified mapping does require some
changes in that area.
I might have forgotten to mention other bits and pieces like the cli or flow on
effects on the UI but that needs to assessed when we have have a design we
agree on. There will be more jiras needed to fix separate parts when the change
is made to the core.
> Allow multiple leaf queues with the same name in CS
> ---------------------------------------------------
>
> Key: YARN-9879
> URL: https://issues.apache.org/jira/browse/YARN-9879
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Gergely Pollak
> Assignee: Gergely Pollak
> Priority: Major
> Attachments: DesignDoc_v1.pdf
>
>
> Currently the leaf queue's name must be unique regardless of its position in
> the queue hierarchy.
> Design doc and first proposal is being made, I'll attach it as soon as it's
> done.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
