[
https://issues.apache.org/jira/browse/IMPALA-14309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Fehr updated IMPALA-14309:
--------------------------------
Description:
By leveraging different rules in the queuePlacementPolicy of the
fair-scheduler.xml file, admission control queues can be auto-selected based on
username/group membership.
Add an example to the [admission control
docs|https://impala.apache.org/docs/build/html/topics/impala_admission_config.html]
explaining how to accomplish this use case.
For example, given this fair-scheduler.xml file:
{noformat}
<?xml version="1.0" encoding="UTF-8"?>
<allocations>
<queue name="root">
<maxResources>200000 mb, 0 vcores</maxResources>
<aclSubmitApps> </aclSubmitApps>
<queue name="produsers">
<maxResources>200000 mb, 0 vcores</maxResources>
<aclSubmitApps> produsers</aclSubmitApps>
</queue>
<queue name="devusers">
<maxResources>200000 mb, 0 vcores</maxResources>
<aclSubmitApps> devusers</aclSubmitApps>
</queue>
<queue name="default">
<maxResources>200000 mb, 0 vcores</maxResources>
<aclSubmitApps>*</aclSubmitApps>
</queue>
</queue>
<queuePlacementPolicy>
<rule name="secondaryGroupExistingQueue" create="false"/>
<rule name="primaryGroup" create="false"/>
<rule name="specified" create="false"/>
<rule name="reject" />
</queuePlacementPolicy>
</allocations>
{noformat}
In this case, leaving REQUEST_POOL unset will automatically direct users in the
produsers group to the produsers queue and users in the devusers group to the
devusers queue. Note: the empty space in the aclSubmitApps setting is
required, otherwise those values will be interpreted as usernames.
If matching only on secondary groups is desired, remove the primaryGroup rule.
If users should not be allowed to specify the REQUEST_POOL config param, remove
the specified rule. If no applicable pool can be found and queries should go
to the default pool, then change the last rule from reject to default.
See the [fair-scheduler.xml
docs|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/FairScheduler.html#Configuration:~:text=A%20queuePlacementPolicy%20element]
for more details.
was:
By leveraging different rules in the queuePlacementPolicy of the
fair-scheduler.xml file, admission control queues can be auto-selected based on
username/group membership.
Add an example to the [admission control
docs|https://impala.apache.org/docs/build/html/topics/impala_admission_config.html]
explaining how to accomplish this use case.
For example, given this fair-scheduler.xml file:
{noformat}
<?xml version="1.0" encoding="UTF-8"?>
<allocations>
<queue name="root">
<maxResources>200000 mb, 0 vcores</maxResources>
<aclSubmitApps> </aclSubmitApps>
<queue name="produsers">
<maxResources>200000 mb, 0 vcores</maxResources>
<aclSubmitApps> produsers</aclSubmitApps>
</queue>
<queue name="devusers">
<maxResources>200000 mb, 0 vcores</maxResources>
<aclSubmitApps> devusers</aclSubmitApps>
</queue>
<queue name="default">
<maxResources>200000 mb, 0 vcores</maxResources>
<aclSubmitApps>*</aclSubmitApps>
</queue>
</queue>
<queuePlacementPolicy>
<rule name="secondaryGroupExistingQueue" create="false"/>
<rule name="primaryGroup" create="false"/>
<rule name="specified" create="false"/>
<rule name="reject" />
</queuePlacementPolicy>
</allocations>
{noformat}
In this case, leaving REQUEST_POOL unset will automatically direct users in the
produsers group to the produsers queue and users in the devusers group to the
devusers queue. Note: the empty space in the aclSubmitApps setting is
required, otherwise those values will be interpreted as usernames.
If matching only on secondary groups is desired, remove the primaryGroup rule.
If users should not be allowed to specify the REQUEST_POOL config param, remove
the specified rule. If no applicable pool can be found and queries should go
to the default pool, then change the last rule from reject to default.
> Create Example for Auto-Choosing Request Pools
> ----------------------------------------------
>
> Key: IMPALA-14309
> URL: https://issues.apache.org/jira/browse/IMPALA-14309
> Project: IMPALA
> Issue Type: Improvement
> Affects Versions: Impala 5.0.0
> Reporter: Jason Fehr
> Priority: Major
>
> By leveraging different rules in the queuePlacementPolicy of the
> fair-scheduler.xml file, admission control queues can be auto-selected based
> on username/group membership.
> Add an example to the [admission control
> docs|https://impala.apache.org/docs/build/html/topics/impala_admission_config.html]
> explaining how to accomplish this use case.
>
> For example, given this fair-scheduler.xml file:
> {noformat}
> <?xml version="1.0" encoding="UTF-8"?>
> <allocations>
> <queue name="root">
> <maxResources>200000 mb, 0 vcores</maxResources>
> <aclSubmitApps> </aclSubmitApps>
> <queue name="produsers">
> <maxResources>200000 mb, 0 vcores</maxResources>
> <aclSubmitApps> produsers</aclSubmitApps>
> </queue>
> <queue name="devusers">
> <maxResources>200000 mb, 0 vcores</maxResources>
> <aclSubmitApps> devusers</aclSubmitApps>
> </queue>
> <queue name="default">
> <maxResources>200000 mb, 0 vcores</maxResources>
> <aclSubmitApps>*</aclSubmitApps>
> </queue>
> </queue>
> <queuePlacementPolicy>
> <rule name="secondaryGroupExistingQueue" create="false"/>
> <rule name="primaryGroup" create="false"/>
> <rule name="specified" create="false"/>
> <rule name="reject" />
> </queuePlacementPolicy>
> </allocations>
> {noformat}
> In this case, leaving REQUEST_POOL unset will automatically direct users in
> the produsers group to the produsers queue and users in the devusers group to
> the devusers queue. Note: the empty space in the aclSubmitApps setting is
> required, otherwise those values will be interpreted as usernames.
> If matching only on secondary groups is desired, remove the primaryGroup
> rule. If users should not be allowed to specify the REQUEST_POOL config
> param, remove the specified rule. If no applicable pool can be found and
> queries should go to the default pool, then change the last rule from reject
> to default.
> See the [fair-scheduler.xml
> docs|https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/FairScheduler.html#Configuration:~:text=A%20queuePlacementPolicy%20element]
> for more details.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
