[
https://issues.apache.org/jira/browse/YARN-3806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589022#comment-14589022
]
Wei Shao commented on YARN-3806:
--------------------------------
Regarding YARN-2986 about unified view of scheduler configuration. I definitely
+1 for having a single unified configuration file.
In addition, to support dynamic created 'single application queue' proposed
here. I suggest to add following feature as well.
In configuration file, each queue can have a subsection to specify the
configuration templates for dynamic created sub queues. And each configuration
template can have an id which applications can reference to.
For example, in configuration file, for a queue with capacity scheduling,
configuration templates with different priorities (template id means priority)
can be specified. The applications submitted to the queue can specify different
priority. The queue will create leaf queue with specified configuration for the
application.
> Proposal of Generic Scheduling Framework for YARN
> -------------------------------------------------
>
> Key: YARN-3806
> URL: https://issues.apache.org/jira/browse/YARN-3806
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: scheduler
> Reporter: Wei Shao
> Attachments: ProposalOfGenericSchedulingFrameworkForYARN-V1.0.pdf,
> ProposalOfGenericSchedulingFrameworkForYARN-V1.1.pdf
>
>
> Currently, a typical YARN cluster runs many different kinds of applications:
> production applications, ad hoc user applications, long running services and
> so on. Different YARN scheduling policies may be suitable for different
> applications. For example, capacity scheduling can manage production
> applications well since application can get guaranteed resource share, fair
> scheduling can manage ad hoc user applications well since it can enforce
> fairness among users. However, current YARN scheduling framework doesn’t have
> a mechanism for multiple scheduling policies work hierarchically in one
> cluster.
> YARN-3306 talked about many issues of today’s YARN scheduling framework, and
> proposed a per-queue policy driven framework. In detail, it supported
> different scheduling policies for leaf queues. However, support of different
> scheduling policies for upper level queues is not seriously considered yet.
> A generic scheduling framework is proposed here to address these limitations.
> It supports different policies for any queue consistently. The proposal tries
> to solve many other issues in current YARN scheduling framework as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
