[jira] [Comment Edited] (YARN-10198) [managedParent].%primary_group mapping rule doesn't work after YARN-9868
[
https://issues.apache.org/jira/browse/YARN-10198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17062531#comment-17062531
]
Peter Bacsko edited comment on YARN-10198 at 3/19/20, 12:39 PM:
I uploaded patch v2.
Couple of things:
1. The most important: I was reasoning about the validation in case of
{{%secondary_group}} and based on the existing code, this cannot have a managed
parent. The queue has to exist, see {{getSecondaryGroup()}}. This also seems to
be in line with Fair Scheduler, where this placement rule is called
"SecondaryGroupExistingPlacementRule". Please confirm this
[~maniraj...@gmail.com], [~prabhujoseph].
2. I had to do some refactor in {{UserGroupMappingPlacementRule}} because
readability is becoming more of a concern with the added features. It will be
addressed by YARN-10199 hopefully.
3. Added extra unit tests. Existing tests are not broken by this change (at
least not the ones in {{TestUserGroupMappingPlacementRule}}).
was (Author: pbacsko):
I uploaded patch v2.
Couple of things:
1. I was reasoning about the validation in case of {{%secondary_group}} and
based on the existing code, this cannot have a managed parent. The queue has to
exist, see {{getSecondaryGroup()}}. This also seems to be in line with Fair
Scheduler, where this placement rule is called
"SecondaryGroupExistingPlacementRule".
2. I had to do some refactor in {{UserGroupMappingPlacementRule}} because
readability is becoming more of a concern with the added features. It will be
addressed by YARN-10199 hopefully.
3. Added extra unit tests. Existing tests are not broken by this change (at
least not the ones in {{TestUserGroupMappingPlacementRule}}).
> [managedParent].%primary_group mapping rule doesn't work after YARN-9868
>
>
> Key: YARN-10198
> URL: https://issues.apache.org/jira/browse/YARN-10198
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-10198-001.patch, YARN-10198-002.patch
>
>
> YARN-9868 introduced an unnecessary check if we have the following placement
> rule:
> [managedParentQueue].%primary_group
> Here, {{%primary_group}} is expected to be created if it doesn't exist.
> However, there is this validation code which is not necessary:
> {noformat}
> } else if (mapping.getQueue().equals(PRIMARY_GROUP_MAPPING)) {
> if (this.queueManager
> .getQueue(groups.getGroups(user).get(0)) != null) {
> return getPlacementContext(mapping,
> groups.getGroups(user).get(0));
> } else {
> return null;
> }
> {noformat}
> We should revert this part to the original version:
> {noformat}
> } else if (mapping.queue.equals(PRIMARY_GROUP_MAPPING)) {
> return getPlacementContext(mapping,
> groups.getGroups(user).get(0));
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10198) [managedParent].%primary_group mapping rule doesn't work after YARN-9868
[
https://issues.apache.org/jira/browse/YARN-10198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17060683#comment-17060683
]
Akhil PB edited comment on YARN-10198 at 3/17/20, 7:40 AM:
---
Expected behaviour:
# *[managedParent].%primary_group*
** Should not create queue named *%primary_group* under *[managedParent]* if
leaf queue named *%primary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%primary_group* does not exists.
# *[managedParent].%secondary_group*
** Should not create queue named *%secondary_group* under *[managedParent]* if
leaf queue named *%secondary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%secondary_group* does not exists.
** If the user does not have *%secondary_group*, then the app should be
submitted to the "default" queue (I guess based on the below static mapping).
Note that for current static mappings like *u:%user:%primary_group* and
*u:%user:%secondary_group*, following is the behavior and works as expected.
# *u:%user:%primary_group*
** App submitted when leaf queue named *%primary_group* is present
** App failed when leaf queue named *%primary_group* is not present or stopped
# *u:%user:%secondary_group*
** App submitted when leaf queue named *%secondary_group* is present
** App submitted to "default" queue when leaf queue named *%secondary_group* is
not present
Please share your thoughts if I am missing something.
cc: [~sunilg] [~prabhujoseph] [~pbacsko]
was (Author: akhilpb):
Expected behavior:
# *[managedParent].%primary_group*
** Should not create queue named *%primary_group* under *[managedParent]* if
leaf queue named *%primary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%primary_group* does not exists.
# *[managedParent].%secondary_group*
** Should not create queue named *%secondary_group* under *[managedParent]* if
leaf queue named *%secondary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%secondary_group* does not exists.
** If the user does not have *%secondary_group*, then the app should be
submitted to the "default" queue (I guess based on the below).
Note that for current static mappings like *u:%user:%primary_group* and
*u:%user:%secondary_group*, following is the behavior and works as expected.
# *u:%user:%primary_group*
** App submitted when leaf queue named *%primary_group* is present
** App failed when leaf queue named *%primary_group* is not present or stopped
# *u:%user:%secondary_group*
** App submitted when leaf queue named *%secondary_group* is present
** App submitted to "default" queue when leaf queue named *%secondary_group* is
not present
Please share your thoughts if I am missing something.
cc: [~sunilg] [~prabhujoseph] [~pbacsko]
> [managedParent].%primary_group mapping rule doesn't work after YARN-9868
>
>
> Key: YARN-10198
> URL: https://issues.apache.org/jira/browse/YARN-10198
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
>
> YARN-9868 introduced an unnecessary check if we have the following placement
> rule:
> [managedParentQueue].%primary_group
> Here, {{%primary_group}} is expected to be created if it doesn't exist.
> However, there is this validation code which is not necessary:
> {noformat}
> } else if (mapping.getQueue().equals(PRIMARY_GROUP_MAPPING)) {
> if (this.queueManager
> .getQueue(groups.getGroups(user).get(0)) != null) {
> return getPlacementContext(mapping,
> groups.getGroups(user).get(0));
> } else {
> return null;
> }
> {noformat}
> We should revert this part to the original version:
> {noformat}
> } else if (mapping.queue.equals(PRIMARY_GROUP_MAPPING)) {
> return getPlacementContext(mapping,
> groups.getGroups(user).get(0));
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10198) [managedParent].%primary_group mapping rule doesn't work after YARN-9868
[
https://issues.apache.org/jira/browse/YARN-10198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17060683#comment-17060683
]
Akhil PB edited comment on YARN-10198 at 3/17/20, 7:34 AM:
---
Expected behavior:
# *[managedParent].%primary_group*
** Should not create queue named *%primary_group* under *[managedParent]* if
leaf queue named *%primary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%primary_group* does not exists.
# *[managedParent].%secondary_group*
** Should not create queue named *%secondary_group* under *[managedParent]* if
leaf queue named *%secondary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%secondary_group* does not exists.
** If the user does not have *%secondary_group*, then the app should be
submitted to the "default" queue (I guess based on the below).
Note that for current static mappings like *u:%user:%primary_group* and
*u:%user:%secondary_group*, following is the behavior and works as expected.
# *u:%user:%primary_group*
** App submitted when leaf queue named *%primary_group* is present
** App failed when leaf queue named *%primary_group* is not present or stopped
# *u:%user:%secondary_group*
** App submitted when leaf queue named *%secondary_group* is present
** App submitted to "default" queue when leaf queue named *%secondary_group* is
not present
Please share your thoughts if I am missing something.
cc: [~sunilg] [~prabhujoseph] [~pbacsko]
was (Author: akhilpb):
Expected behavior:
# *[managedParent].%primary_group*
** Should not create queue named *%primary_group* under *[managedParent]* if
leaf queue named *%primary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%primary_group* does not exists.
# *[managedParent].%secondary_group*
** Should not create queue named *%secondary_group* under *[managedParent]* if
leaf queue named *%secondary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%secondary_group* does not exists.
Note that for current static mappings like *u:%user:%primary_group* and
*u:%user:%secondary_group*, following is the behavior and works as expected.
# *u:%user:%primary_group*
** App submitted when leaf queue named *%primary_group* is present
** App failed when leaf queue named *%primary_group* is not present or stopped
# *u:%user:%secondary_group*
** App submitted when leaf queue named *%secondary_group* is present
** App submitted to "default" queue when leaf queue named *%secondary_group* is
not present
Please share your thoughts if I am missing something.
cc: [~sunilg] [~prabhujoseph] [~pbacsko]
> [managedParent].%primary_group mapping rule doesn't work after YARN-9868
>
>
> Key: YARN-10198
> URL: https://issues.apache.org/jira/browse/YARN-10198
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
>
> YARN-9868 introduced an unnecessary check if we have the following placement
> rule:
> [managedParentQueue].%primary_group
> Here, {{%primary_group}} is expected to be created if it doesn't exist.
> However, there is this validation code which is not necessary:
> {noformat}
> } else if (mapping.getQueue().equals(PRIMARY_GROUP_MAPPING)) {
> if (this.queueManager
> .getQueue(groups.getGroups(user).get(0)) != null) {
> return getPlacementContext(mapping,
> groups.getGroups(user).get(0));
> } else {
> return null;
> }
> {noformat}
> We should revert this part to the original version:
> {noformat}
> } else if (mapping.queue.equals(PRIMARY_GROUP_MAPPING)) {
> return getPlacementContext(mapping,
> groups.getGroups(user).get(0));
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10198) [managedParent].%primary_group mapping rule doesn't work after YARN-9868
[
https://issues.apache.org/jira/browse/YARN-10198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17060683#comment-17060683
]
Akhil PB edited comment on YARN-10198 at 3/17/20, 7:30 AM:
---
Expected behavior:
# *[managedParent].%primary_group*
** Should not create queue named *%primary_group* under *[managedParent]* if
leaf queue named *%primary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%primary_group* does not exists.
# *[managedParent].%secondary_group*
** Should not create queue named *%secondary_group* under *[managedParent]* if
leaf queue named *%secondary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%secondary_group* does not exists.
Note that for current static mappings like *u:%user:%primary_group* and
*u:%user:%secondary_group*, following is the behavior and works as expected.
# *u:%user:%primary_group*
** App submitted when leaf queue named *%primary_group* is present
** App failed when leaf queue named *%primary_group* is not present or stopped
# *u:%user:%secondary_group*
** App submitted when leaf queue named *%secondary_group* is present
** App submitted to "default" queue when leaf queue named *%secondary_group* is
not present
Please share your thoughts if I am missing something.
cc: [~sunilg] [~prabhujoseph] [~pbacsko]
was (Author: akhilpb):
Expected behavior:
# *[managedParent].%primary_group*
** Should not create queue named *%primary_group* under *[managedParent]* if
leaf queue named *%primary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%primary_group* does not exists.
# *[managedParent].%secondary_group*
** Should not create queue named *%secondary_group* under *[managedParent]* if
leaf queue named *%secondary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%secondary_group* does not exists.
Note that for current static mappings like *u:%user:%primary_group* and
*u:%user:%secondary_group*, following is the behavior and works as expected.
# *u:%user:%primary_group*
** App submitted when leaf queue named *%primary_group* is present
** App failed when leaf queue named *%primary_group* is not present or stopped
# *u:%user:%secondary_group*
** App submitted when leaf queue named *%secondary_group* is present
** App submitted to "default" queue when leaf queue named *%secondary_group* is
not present
Please share your thoughts if I am missing something.
cc: [~sunilg] [~prabhujoseph] [~pbacsko]
> [managedParent].%primary_group mapping rule doesn't work after YARN-9868
>
>
> Key: YARN-10198
> URL: https://issues.apache.org/jira/browse/YARN-10198
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
>
> YARN-9868 introduced an unnecessary check if we have the following placement
> rule:
> [managedParentQueue].%primary_group
> Here, {{%primary_group}} is expected to be created if it doesn't exist.
> However, there is this validation code which is not necessary:
> {noformat}
> } else if (mapping.getQueue().equals(PRIMARY_GROUP_MAPPING)) {
> if (this.queueManager
> .getQueue(groups.getGroups(user).get(0)) != null) {
> return getPlacementContext(mapping,
> groups.getGroups(user).get(0));
> } else {
> return null;
> }
> {noformat}
> We should revert this part to the original version:
> {noformat}
> } else if (mapping.queue.equals(PRIMARY_GROUP_MAPPING)) {
> return getPlacementContext(mapping,
> groups.getGroups(user).get(0));
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10198) [managedParent].%primary_group mapping rule doesn't work after YARN-9868
[
https://issues.apache.org/jira/browse/YARN-10198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17060683#comment-17060683
]
Akhil PB edited comment on YARN-10198 at 3/17/20, 7:30 AM:
---
Expected behavior:
# *[managedParent].%primary_group*
** Should not create queue named *%primary_group* under *[managedParent]* if
leaf queue named *%primary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%primary_group* does not exists.
# *[managedParent].%secondary_group*
** Should not create queue named *%secondary_group* under *[managedParent]* if
leaf queue named *%secondary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%secondary_group* does not exists.
Note that for current static mappings like *u:%user:%primary_group* and
*u:%user:%secondary_group*, following is the behavior and works as expected.
# *u:%user:%primary_group*
** App submitted when leaf queue named *%primary_group* is present
** App failed when leaf queue named *%primary_group* is not present or stopped
# *u:%user:%secondary_group*
** App submitted when leaf queue named *%secondary_group* is present
** App submitted to "default" queue when leaf queue named *%secondary_group* is
not present
Please share your thoughts if I am missing something.
cc: [~sunilg] [~prabhujoseph] [~pbacsko]
was (Author: akhilpb):
Expected behavior:
# *[managedParent].%primary_group*
** Should not create queue named *%primary_group* under *[managedParent]* if
leaf queue named *%primary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%primary_group* does not exists.
# *[managedParent].%secondary_group*
** Should not create queue named *%secondary_group* under *[managedParent]* if
leaf queue named *%secondary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%secondary_group* does not exists.
Note that for current static mappings like *u:%user:%primary_group* and
*u:%user:%secondary_group*, following is the behavior and works as expected.
# *u:%user:%primary_group*
** App submitted when leaf queue named *%primary_group* is present
** App failed when leaf queue named *%primary_group* is not present or stopped
# *u:%user:%secondary_group*
** App submitted when leaf queue named *%secondary_group* is present
** App submitted to "default" queue when leaf queue named *%secondary_group* is
not present
Please share your thoughts if I am missing something.
cc: [~sunilg] [~prabhujoseph] [~pbacsko]
> [managedParent].%primary_group mapping rule doesn't work after YARN-9868
>
>
> Key: YARN-10198
> URL: https://issues.apache.org/jira/browse/YARN-10198
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
>
> YARN-9868 introduced an unnecessary check if we have the following placement
> rule:
> [managedParentQueue].%primary_group
> Here, {{%primary_group}} is expected to be created if it doesn't exist.
> However, there is this validation code which is not necessary:
> {noformat}
> } else if (mapping.getQueue().equals(PRIMARY_GROUP_MAPPING)) {
> if (this.queueManager
> .getQueue(groups.getGroups(user).get(0)) != null) {
> return getPlacementContext(mapping,
> groups.getGroups(user).get(0));
> } else {
> return null;
> }
> {noformat}
> We should revert this part to the original version:
> {noformat}
> } else if (mapping.queue.equals(PRIMARY_GROUP_MAPPING)) {
> return getPlacementContext(mapping,
> groups.getGroups(user).get(0));
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10198) [managedParent].%primary_group mapping rule doesn't work after YARN-9868
[
https://issues.apache.org/jira/browse/YARN-10198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17060683#comment-17060683
]
Akhil PB edited comment on YARN-10198 at 3/17/20, 7:29 AM:
---
Expected behavior:
# *[managedParent].%primary_group*
** Should not create queue named *%primary_group* under *[managedParent]* if
leaf queue named *%primary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%primary_group* does not exists.
# *[managedParent].%secondary_group*
** Should not create queue named *%secondary_group* under *[managedParent]* if
leaf queue named *%secondary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%secondary_group* does not exists.
Note that for current static mappings like *u:%user:%primary_group* and
*u:%user:%secondary_group*, following is the behavior and works as expected.
# *u:%user:%primary_group*
** App submitted when leaf queue named *%primary_group*( is present
** App failed when leaf queue named *%primary_group* is not present or stopped
# *u:%user:%secondary_group*
** App submitted when leaf queue named *%secondary_group* is present
** App submitted to "default" queue when leaf queue named *%secondary_group* is
not present
Please share your thoughts if I am missing something.
cc: [~sunilg] [~prabhujoseph] [~pbacsko]
was (Author: akhilpb):
Expected behavior:
# *[managedParent].%primary_group*
** Should not create queue name *%primary_group* under *[managedParent]* if
leaf queue named *%primary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%primary_group* does not exists.
# *[managedParent].%secondary_group*
** Should not create queue name *%secondary_group* under *[managedParent]* if
leaf queue named *%secondary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%secondary_group* does not exists.
Note that for current static mappings like *u:%user:%primary_group* and
*u:%user:%secondary_group*, following is the behavior and works as expected.
# *u:%user:%primary_group*
** App submitted when leaf queue named *%primary_group*( is present
** App failed when leaf queue named *%primary_group* is not present or stopped
# *u:%user:%secondary_group*
** App submitted when leaf queue named *%secondary_group* is present
** App submitted to "default" queue when leaf queue named *%secondary_group* is
not present
Please share your thoughts if I am missing something.
cc: [~sunilg] [~prabhujoseph] [~pbacsko]
> [managedParent].%primary_group mapping rule doesn't work after YARN-9868
>
>
> Key: YARN-10198
> URL: https://issues.apache.org/jira/browse/YARN-10198
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
>
> YARN-9868 introduced an unnecessary check if we have the following placement
> rule:
> [managedParentQueue].%primary_group
> Here, {{%primary_group}} is expected to be created if it doesn't exist.
> However, there is this validation code which is not necessary:
> {noformat}
> } else if (mapping.getQueue().equals(PRIMARY_GROUP_MAPPING)) {
> if (this.queueManager
> .getQueue(groups.getGroups(user).get(0)) != null) {
> return getPlacementContext(mapping,
> groups.getGroups(user).get(0));
> } else {
> return null;
> }
> {noformat}
> We should revert this part to the original version:
> {noformat}
> } else if (mapping.queue.equals(PRIMARY_GROUP_MAPPING)) {
> return getPlacementContext(mapping,
> groups.getGroups(user).get(0));
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10198) [managedParent].%primary_group mapping rule doesn't work after YARN-9868
[
https://issues.apache.org/jira/browse/YARN-10198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17060683#comment-17060683
]
Akhil PB edited comment on YARN-10198 at 3/17/20, 7:29 AM:
---
Expected behavior:
# *[managedParent].%primary_group*
** Should not create queue named *%primary_group* under *[managedParent]* if
leaf queue named *%primary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%primary_group* does not exists.
# *[managedParent].%secondary_group*
** Should not create queue named *%secondary_group* under *[managedParent]* if
leaf queue named *%secondary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%secondary_group* does not exists.
Note that for current static mappings like *u:%user:%primary_group* and
*u:%user:%secondary_group*, following is the behavior and works as expected.
# *u:%user:%primary_group*
** App submitted when leaf queue named *%primary_group* is present
** App failed when leaf queue named *%primary_group* is not present or stopped
# *u:%user:%secondary_group*
** App submitted when leaf queue named *%secondary_group* is present
** App submitted to "default" queue when leaf queue named *%secondary_group* is
not present
Please share your thoughts if I am missing something.
cc: [~sunilg] [~prabhujoseph] [~pbacsko]
was (Author: akhilpb):
Expected behavior:
# *[managedParent].%primary_group*
** Should not create queue named *%primary_group* under *[managedParent]* if
leaf queue named *%primary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%primary_group* does not exists.
# *[managedParent].%secondary_group*
** Should not create queue named *%secondary_group* under *[managedParent]* if
leaf queue named *%secondary_group* already exists.
** Should create new child queue under *[managedParent]* if leaf queue named
*%secondary_group* does not exists.
Note that for current static mappings like *u:%user:%primary_group* and
*u:%user:%secondary_group*, following is the behavior and works as expected.
# *u:%user:%primary_group*
** App submitted when leaf queue named *%primary_group*( is present
** App failed when leaf queue named *%primary_group* is not present or stopped
# *u:%user:%secondary_group*
** App submitted when leaf queue named *%secondary_group* is present
** App submitted to "default" queue when leaf queue named *%secondary_group* is
not present
Please share your thoughts if I am missing something.
cc: [~sunilg] [~prabhujoseph] [~pbacsko]
> [managedParent].%primary_group mapping rule doesn't work after YARN-9868
>
>
> Key: YARN-10198
> URL: https://issues.apache.org/jira/browse/YARN-10198
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
>
> YARN-9868 introduced an unnecessary check if we have the following placement
> rule:
> [managedParentQueue].%primary_group
> Here, {{%primary_group}} is expected to be created if it doesn't exist.
> However, there is this validation code which is not necessary:
> {noformat}
> } else if (mapping.getQueue().equals(PRIMARY_GROUP_MAPPING)) {
> if (this.queueManager
> .getQueue(groups.getGroups(user).get(0)) != null) {
> return getPlacementContext(mapping,
> groups.getGroups(user).get(0));
> } else {
> return null;
> }
> {noformat}
> We should revert this part to the original version:
> {noformat}
> } else if (mapping.queue.equals(PRIMARY_GROUP_MAPPING)) {
> return getPlacementContext(mapping,
> groups.getGroups(user).get(0));
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
