[
https://issues.apache.org/jira/browse/YARN-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145746#comment-16145746
]
Wangda Tan commented on YARN-7117:
----------------------------------
Thanks [~asuresh] / [~jlowe],
Both of you commented on unused queues. IIRC, in fair scheduler, there's no
guaranteed resource concept. minResource affect how queues sharing resource,
but it is not a guaranteed value since it allows
- {{Σ(child.minResource) > parent.minResource.}}
I'm agree what [~jlowe] mentioned:
bq. In short, if we're going to allow new, previously illegal configurations
for auto-queues then arguably we should be consistent and allow them elsewhere
as well.
To make behavior of automatically created queue consistent with manual created
queues, it might be problematic if we allow overcommit of auto created queue
but not allow overcommit of manual queues.
Adding queue with zero guaranteed resource is one possible solution, but it may
not be good enough for resource sharing since many users don't frequently use
cluster.
Instead, maybe admin can configure a "delete-policy" which delete queues which
are unused for X secs. Thoughts?
> Capacity Scheduler: Support Auto Creation of Leaf Queues While Doing Queue
> Mapping
> ----------------------------------------------------------------------------------
>
> Key: YARN-7117
> URL: https://issues.apache.org/jira/browse/YARN-7117
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: capacity scheduler
> Reporter: Wangda Tan
> Assignee: Wangda Tan
>
> Currently Capacity Scheduler doesn't support auto creation of queues when
> doing queue mapping. We saw more and more use cases which has complex queue
> mapping policies configured to handle application to queues mapping.
> The most common use case of CapacityScheduler queue mapping is to create one
> queue for each user/group. However update {{capacity-scheduler.xml}} and
> {{RMAdmin:refreshQueues}} needs to be done when new user/group onboard. One
> of the option to solve the problem is automatically create queues when new
> user/group arrives.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
