[
https://issues.apache.org/jira/browse/YARN-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16147241#comment-16147241
]
Jason Lowe commented on YARN-7117:
----------------------------------
bq. This is a valid concern, instead of deleting queue, how about stopping the
queue?
Probably a bit better depending upon the use case. Still may be surprising to
a user who is submitting regularly (e.g.: via Oozie) and suddenly their
submissions fail because the queue is stopped due to some other users suddenly
submitting some jobs at an inopportune time and forcing this user's queue to
stop as a result. I guess that's not a valid use case for this. As long as
it's clear how it behaves and that matches expectations of users looking to use
this feature I guess it's OK.
It may be even better to just leave them as-is and require someone to cleanup
unused queues in order to allow more auto-queues, otherwise I'm not sure we
could ever deploy this. It would allow one user to remove capabilities
previously given to another user which is a denial of service. In that sense
it seems better to be up front and either say, "no auto queue for you until
someone frees up space" or "hey, here's a queue with little-to-no guarantees
but it could have a lot of capacity depending upon other user activity." That
latter model (as a single queue for us but could be a bunch of separate,
per-user queues if necessary) has worked pretty well for us for ad-hoc jobs.
> Capacity Scheduler: Support Auto Creation of Leaf Queues While Doing Queue
> Mapping
> ----------------------------------------------------------------------------------
>
> Key: YARN-7117
> URL: https://issues.apache.org/jira/browse/YARN-7117
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: capacity scheduler
> Reporter: Wangda Tan
> Assignee: Wangda Tan
>
> Currently Capacity Scheduler doesn't support auto creation of queues when
> doing queue mapping. We saw more and more use cases which has complex queue
> mapping policies configured to handle application to queues mapping.
> The most common use case of CapacityScheduler queue mapping is to create one
> queue for each user/group. However update {{capacity-scheduler.xml}} and
> {{RMAdmin:refreshQueues}} needs to be done when new user/group onboard. One
> of the option to solve the problem is automatically create queues when new
> user/group arrives.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
