[ https://issues.apache.org/jira/browse/YARN-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158854#comment-16158854 ]
Clay B. commented on YARN-7117: ------------------------------- As to my use-case: I have an ad-hoc "users" queue which is a best-effort queue with some minimum capacity granted to that overall queue. E.g. users can get their 4 vcpus and 4GB of RAM (or more if available), if the users best-effort queue is not over-subscribed then they can pre-empt jobs running outside that queue for their minimum. However, on our Dev systems we may have 1,000+ developers and certainly we don't expect them all to run at once (e.g. typical teleco-style over-subscription assumptions) but we do want to ensure that other queues' guaranteed capacity are not impacted and we want to ensure we have per-user metrics on ad-hoc jobs which individual queues provide us via JMX over having a single ad-hoc queue with YARN per-user metrics overall (where other queues may be prudently used by a user and analysed). Should a user submit when the parent queue is saturated it would be due to a massive influx of users since, for me, one user has a minimum of 1% of the overall queue capacity. > Capacity Scheduler: Support Auto Creation of Leaf Queues While Doing Queue > Mapping > ---------------------------------------------------------------------------------- > > Key: YARN-7117 > URL: https://issues.apache.org/jira/browse/YARN-7117 > Project: Hadoop YARN > Issue Type: New Feature > Components: capacity scheduler > Reporter: Wangda Tan > Assignee: Wangda Tan > > Currently Capacity Scheduler doesn't support auto creation of queues when > doing queue mapping. We saw more and more use cases which has complex queue > mapping policies configured to handle application to queues mapping. > The most common use case of CapacityScheduler queue mapping is to create one > queue for each user/group. However update {{capacity-scheduler.xml}} and > {{RMAdmin:refreshQueues}} needs to be done when new user/group onboard. One > of the option to solve the problem is automatically create queues when new > user/group arrives. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org