[
https://issues.apache.org/jira/browse/YARN-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szilard Nemeth updated YARN-11069:
----------------------------------
Description:
During the investigation YARN-10922 I found multiple issues with dynamic ACL
handling, no wonder it was documented as unsupported. But in some cases it did
work and some users utilised that behaviour.
There is a severe bug when using Flexible AQC and dynamic parent queues,
*anybody* can submit an app regardless of what ACLs are set (when the dynamic
parent queue is not exists yet) - see: YARN-11066.
After this dynamic ACL feature YARN-11060 should be fixed as well.
h4. Legacy AQC:
{code:java}
yarn.scheduler.capacity.root.managed.auto-create-child-queue.enabled: true
yarn.scheduler.capacity.root.managed.acl_submit_applications: nobody
yarn.scheduler.capacity.root.managed.acl_administer_queue: nobody
yarn.scheduler.capacity.root.managed.acl_submit_applications: user1
yarn.scheduler.capacity.root.managed.acl_administer_queue: admin1
yarn.scheduler.capacity.root.managed.leaf-queue-template.acl_submit_applications:
user2
yarn.scheduler.capacity.root.managed.leaf-queue-template.acl_administer_queue:
admin2
{code}
user2 can submit an application to root.managed.user2 even if the queue doesn't
exist yet due to template
*Permissions:*
* root: \{nobody, nobody}
* root.managed: \{user1, admin1}
* root.managed.auto: \{user2, admin2}
h4. Flexible AQC:
{code:java}
yarn.scheduler.capacity.root.auto-queue-creation-v2.enabled: true
yarn.scheduler.capacity.root.acl_submit_applications: user
yarn.scheduler.capacity.root.acl_administer_queue: admin
yarn.scheduler.capacity.root.auto-queue-creation-v2.parent-template.acl_submit_applications:
parentUser1
yarn.scheduler.capacity.root.auto-queue-creation-v2.parent-template.acl_administer_queue:
parentAdmin1
yarn.scheduler.capacity.root.*.auto-queue-creation-v2.leaf-template.acl_submit_applications:
user1
yarn.scheduler.capacity.root.*.auto-queue-creation-v2.leaf-template.acl_administer_queue:
admin1
{code}
user1 can submit an application to root.autoParent.user1 even if the queue
doesn't exist yet due to template
*Permissions:*
* root: \{user, admin}
* root.autoParent: \{parentUser1, parentAdmin1}
* root.autoParent.autoLeaf: \{user1, admin1}
NOTE: the *.leaf-template* and *.parent-template* overwrites the *.template*
variant.
The Scheduler Response should contain the correct values for the queueAcls
field for the dynamic queues.
The wildcard pattern for flexible AQC was not working properly for root.* queue
pattern.
was:
During the investigation YARN-10922 I found multiple issues with dynamic ACL
handling, no wonder it was documented as unsupported. But in some cases it did
work and some users utilised that behaviour.
There is a severe bug when using Flexible AQC and dynamic parent queues,
*anybody* can submit an app regardless of what ACLs are set (when the dynamic
parent queue is not exists yet) - see: YARN-11066.
After this dynamic ACL feature YARN-11060 should be fixed as well.
h4. Legacy AQC:
{code:java}
yarn.scheduler.capacity.root.managed.auto-create-child-queue.enabled: true
yarn.scheduler.capacity.root.managed.acl_submit_applications: nobody
yarn.scheduler.capacity.root.managed.acl_administer_queue: nobody
yarn.scheduler.capacity.root.managed.acl_submit_applications: user1
yarn.scheduler.capacity.root.managed.acl_administer_queue: admin1
yarn.scheduler.capacity.root.managed.leaf-queue-template.acl_submit_applications:
user2
yarn.scheduler.capacity.root.managed.leaf-queue-template.acl_administer_queue:
admin2
{code}
user2 can submit an application to root.managed.user2 even if the queue doesn't
exist yet due to template
*Permissions:*
* root: \{nobody, nobody}
* root.managed: \{user1, admin1}
* root.managed.auto: \{user2, admin2}
h4.
Flexible AQC:
{code:java}
yarn.scheduler.capacity.root.auto-queue-creation-v2.enabled: true
yarn.scheduler.capacity.root.acl_submit_applications: user
yarn.scheduler.capacity.root.acl_administer_queue: admin
yarn.scheduler.capacity.root.auto-queue-creation-v2.parent-template.acl_submit_applications:
parentUser1
yarn.scheduler.capacity.root.auto-queue-creation-v2.parent-template.acl_administer_queue:
parentAdmin1
yarn.scheduler.capacity.root.*.auto-queue-creation-v2.leaf-template.acl_submit_applications:
user1
yarn.scheduler.capacity.root.*.auto-queue-creation-v2.leaf-template.acl_administer_queue:
admin1
{code}
user1 can submit an application to root.autoParent.user1 even if the queue
doesn't exist yet due to template
*Permissions:*
* root: \{user, admin}
* root.autoParent: \{parentUser1, parentAdmin1}
* root.autoParent.autoLeaf: \{user1, admin1}
NOTE: the *.leaf-template* and *.parent-template* overwrites the *.template*
variant.
The Scheduler Response should contain the correct values for the queueAcls
field for the dynamic queues.
The wildcard pattern for flexible AQC was not working properly for root.* queue
pattern.
> Dynamic Queue ACL handling in Legacy and Flexible Auto Created Queues
> ---------------------------------------------------------------------
>
> Key: YARN-11069
> URL: https://issues.apache.org/jira/browse/YARN-11069
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: capacity scheduler, yarn
> Reporter: Tamas Domok
> Assignee: Tamas Domok
> Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> During the investigation YARN-10922 I found multiple issues with dynamic ACL
> handling, no wonder it was documented as unsupported. But in some cases it
> did work and some users utilised that behaviour.
> There is a severe bug when using Flexible AQC and dynamic parent queues,
> *anybody* can submit an app regardless of what ACLs are set (when the dynamic
> parent queue is not exists yet) - see: YARN-11066.
> After this dynamic ACL feature YARN-11060 should be fixed as well.
>
> h4. Legacy AQC:
> {code:java}
> yarn.scheduler.capacity.root.managed.auto-create-child-queue.enabled: true
> yarn.scheduler.capacity.root.managed.acl_submit_applications: nobody
> yarn.scheduler.capacity.root.managed.acl_administer_queue: nobody
> yarn.scheduler.capacity.root.managed.acl_submit_applications: user1
> yarn.scheduler.capacity.root.managed.acl_administer_queue: admin1
> yarn.scheduler.capacity.root.managed.leaf-queue-template.acl_submit_applications:
> user2
> yarn.scheduler.capacity.root.managed.leaf-queue-template.acl_administer_queue:
> admin2
> {code}
> user2 can submit an application to root.managed.user2 even if the queue
> doesn't exist yet due to template
>
> *Permissions:*
> * root: \{nobody, nobody}
> * root.managed: \{user1, admin1}
> * root.managed.auto: \{user2, admin2}
> h4. Flexible AQC:
> {code:java}
> yarn.scheduler.capacity.root.auto-queue-creation-v2.enabled: true
> yarn.scheduler.capacity.root.acl_submit_applications: user
> yarn.scheduler.capacity.root.acl_administer_queue: admin
> yarn.scheduler.capacity.root.auto-queue-creation-v2.parent-template.acl_submit_applications:
> parentUser1
> yarn.scheduler.capacity.root.auto-queue-creation-v2.parent-template.acl_administer_queue:
> parentAdmin1
> yarn.scheduler.capacity.root.*.auto-queue-creation-v2.leaf-template.acl_submit_applications:
> user1
> yarn.scheduler.capacity.root.*.auto-queue-creation-v2.leaf-template.acl_administer_queue:
> admin1
> {code}
> user1 can submit an application to root.autoParent.user1 even if the queue
> doesn't exist yet due to template
> *Permissions:*
> * root: \{user, admin}
> * root.autoParent: \{parentUser1, parentAdmin1}
> * root.autoParent.autoLeaf: \{user1, admin1}
>
> NOTE: the *.leaf-template* and *.parent-template* overwrites the *.template*
> variant.
>
>
> The Scheduler Response should contain the correct values for the queueAcls
> field for the dynamic queues.
> The wildcard pattern for flexible AQC was not working properly for root.*
> queue pattern.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
