[
https://issues.apache.org/jira/browse/YARN-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145916#comment-16145916
]
Wangda Tan commented on YARN-7117:
----------------------------------
Thanks [[email protected]] for suggestions, all good points,
bq. These dynamic queues are created under any queue ? how to map which queue
this new dynamic queue is created?
This depends on mapping policy and if the parent queue support auto create sub
queues. (See my first comment).
bq. i was actually thinking whether we can use reservation queues under default
queues which is not time bounded but will be alive till the apps are running
under it. (though not completely thought about approach)
This makes a special queue to support the case, I'm not sure if we want to go
the direction or not. I would prefer to make auto created queues same as manual
created queues.
bq. Queue should be created based on user name user group name on what basis
will it be decided?
This depends on {{yarn.scheduler.capacity.queue-mappings}} settings, and in
addition, after YARN-3635/YARN-6689, pluggable queue policy can be specified as
well.
bq. For ACL would it be simpler to just have Submit-ACL based on user if the
queue is created based on user and user group if created based on group and
Admin ACLs inherited from the parent ?
This is definitely a good starting point. To better support
YARN-3635/YARN-6689, admin may want to have more comprehensive syntaxes to
specify ACL.
bq. based on existing configuration i don't think we can achieve it (correct me
if i am wrong)
Yes it will be a new config, actually I'm not sure if it is really need.
Probably we can create queue under a parent if the parent has enough
un-assigned guaranteed resources.
> Capacity Scheduler: Support Auto Creation of Leaf Queues While Doing Queue
> Mapping
> ----------------------------------------------------------------------------------
>
> Key: YARN-7117
> URL: https://issues.apache.org/jira/browse/YARN-7117
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: capacity scheduler
> Reporter: Wangda Tan
> Assignee: Wangda Tan
>
> Currently Capacity Scheduler doesn't support auto creation of queues when
> doing queue mapping. We saw more and more use cases which has complex queue
> mapping policies configured to handle application to queues mapping.
> The most common use case of CapacityScheduler queue mapping is to create one
> queue for each user/group. However update {{capacity-scheduler.xml}} and
> {{RMAdmin:refreshQueues}} needs to be done when new user/group onboard. One
> of the option to solve the problem is automatically create queues when new
> user/group arrives.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
