[
https://issues.apache.org/jira/browse/YARN-9841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940924#comment-16940924
]
Peter Bacsko edited comment on YARN-9841 at 9/30/19 12:28 PM:
--------------------------------------------------------------
I checked the suspected mapping bugs:
{{"u:user2:%primary_group"}} - I can confirm that it's not working. The
returned queue is "%primary_group", so resolving the primary group for the user
does not take place.
"_Use case of "u:%user:parentqueue.%user" mapping doesn't return expected o/p
when it is working in conjunction with "u:%user:%primary_group" mapping. Where
as, Using "u:%user:parentqueue.%user" mapping alone is working as expected."_ -
I think this is not a bug, at least not they way you're suggesting. The order
of mapping rules matter. If your example, "u:%user:%primary_group" simply has
precedence and it "shadows" all other user-related mapping that might follow.
Here, the problem is that end-users are not notified about this. Fair-scheduler
checks whether a placement rule is terminal and whether there are subsequent
rules. Such a configuration is invalid because the remaining rules will always
be ignored. CS, on the other hand, does not care about this and just simply
returns the first matching rule, not checking the remaining ones.
was (Author: pbacsko):
I checked the first rule mapping:
{{"u:user2:%primary_group"}} - I can confirm that it's not working. The
returned queue is "%primary_group", so resolving the primary group for the user
does not take place.
"_Use case of "u:%user:parentqueue.%user" mapping doesn't return expected o/p
when it is working in conjunction with "u:%user:%primary_group" mapping. Where
as, Using "u:%user:parentqueue.%user" mapping alone is working as expected."_ -
I think this is not a bug, at least not they way you're suggesting. The order
of mapping rules matter. If your example, "u:%user:%primary_group" simply has
precedence and it "shadows" all other user-related mapping that might follow.
Here, the problem is that end-users are not notified about this. Fair-scheduler
checks whether a placement rule is terminal and whether there are subsequent
rules. Such a configuration is invalid because the remaining rules will always
be ignored. CS, on the other hand, does not care about this and just simply
returns the first matching rule, not checking the remaining ones.
> Capacity scheduler: add support for combined %user + %primary_group mapping
> ---------------------------------------------------------------------------
>
> Key: YARN-9841
> URL: https://issues.apache.org/jira/browse/YARN-9841
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler
> Reporter: Peter Bacsko
> Assignee: Manikandan R
> Priority: Major
> Attachments: YARN-9841.001.patch, YARN-9841.001.patch,
> YARN-9841.002.patch, YARN-9841.junit.patch
>
>
> Right now in CS, using {{%primary_group}} with a parent queue is only
> possible this way:
> {{u:%user:parentqueue.%primary_group}}
> Looking at
> https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/placement/UserGroupMappingPlacementRule.java,
> we cannot do something like:
> {{u:%user:%primary_group.%user}}
> Fair Scheduler supports a nested rule where such a placement/mapping rule is
> possible. This improvement would reduce this feature gap.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
