[
https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17015701#comment-17015701
]
Wilfred Spiegelenburg commented on YARN-9879:
---------------------------------------------
A per queue flag looks very strange. I am also not sure it will help or add
anything on top of having a global flag that just prevents the config change.
Summarising the proposed solution: add a flag that prevents the admin from
adding non unique leaf queue names and thus fail the config change when he/she
tries
The behaviour inside the scheduler must all be based on the full queue paths
anyway. You cannot have one queue being addressed by the leaf name and the
other by the path. The code complexity to do that would be enormous and lead to
unsupportable code. That means that after the placement rule(s) are run and the
app is placed everything must be based on a full path.
Placement rules throw up a totally different issue here. When we use placement
rules we have one of two possible cases:
* the rule generates a queue name and a parent queue name, i.e. a path
* the rule generates just a leaf queue name
Which means that the rule can generate a leaf queue anywhere in the hierarchy
without specifying a hierarchy. So no parent is set by the rule but the leaf
queue generated could be located below a parent. With that last possibility we
have the extra complexity in that the rules are not behaving consistently.
Example:
Two CS definitions to compare both allow queue creation and overwrite of the
submitted queue:
# queues: root.parent.wilfred
# queues: root
mapping rule defined: {{u:%user:%user}}
1) user submitting the app is {{wilfred}} queue given on submission is default
In CS config 1 we submit to the {{root.parent.wilfred}} queue while in the
second CS config we submit to {{root.wilfred}} queue.
2) user submitting the app is {{peter}} queue given on submission is default
In both CS configs we submit to the {{root.peter}} queue.
With different config at the CS level but for the same rule we place the app in
a sub queue sometimes but not the other, that is inconsistent.
I think rules even need to start taking this flag into account to preserve this
inconsistent behaviour.
> Allow multiple leaf queues with the same name in CS
> ---------------------------------------------------
>
> Key: YARN-9879
> URL: https://issues.apache.org/jira/browse/YARN-9879
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Gergely Pollak
> Assignee: Gergely Pollak
> Priority: Major
> Attachments: DesignDoc_v1.pdf
>
>
> Currently the leaf queue's name must be unique regardless of its position in
> the queue hierarchy.
> Design doc and first proposal is being made, I'll attach it as soon as it's
> done.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
