[
https://issues.apache.org/jira/browse/YARN-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16148040#comment-16148040
]
Carlo Curino commented on YARN-7117:
------------------------------------
I have very skimmed the thread very briefly, but it appears to me that you are
looking at lots of the problems are similar to those solved by the
{{PlanQueue}} {{ReservationQueue}} and {{PlanFollower}}.
There we dealt with dynamic creation of queues, adding up to 100% as well as we
can provide (thanks to the planning aspects) stronger guarantees for the Oozie
like use cases (recurring aspects of this in YARN-5326).
I would ask you to look at that stuff, and figure out whether you can leverage
some/most of it, instead of building a parallel solution.
Overall the {{PlanFollower}} design of "observe an external signal and publish
it to scheduler" seems clean and powerful, as it decouples the CS inner
workings from outside (slower evolving) phenomena.
> Capacity Scheduler: Support Auto Creation of Leaf Queues While Doing Queue
> Mapping
> ----------------------------------------------------------------------------------
>
> Key: YARN-7117
> URL: https://issues.apache.org/jira/browse/YARN-7117
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: capacity scheduler
> Reporter: Wangda Tan
> Assignee: Wangda Tan
>
> Currently Capacity Scheduler doesn't support auto creation of queues when
> doing queue mapping. We saw more and more use cases which has complex queue
> mapping policies configured to handle application to queues mapping.
> The most common use case of CapacityScheduler queue mapping is to create one
> queue for each user/group. However update {{capacity-scheduler.xml}} and
> {{RMAdmin:refreshQueues}} needs to be done when new user/group onboard. One
> of the option to solve the problem is automatically create queues when new
> user/group arrives.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
