[
https://issues.apache.org/jira/browse/YARN-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257288#comment-14257288
]
Vinod Kumar Vavilapalli commented on YARN-2986:
-----------------------------------------------
Today's scheduler configuration is fragmented and non-intuitive, and needs to
be improved.
- Each scheduler has its own configuration file and schema
- Each scheduler maintains its own custom configuration parser
- Though there is a first-class notion of many things, the configurations are
duplicated. For e.g queues are first-class, but configuration queue-acls is per
scheduler
- Scheduler configuration is split across yarn-site.xml and
capacity-scheduler/fair-scheduler files
- Queues have a notion of hierarchy, but the configuration is a flat
dictionary. This is very painful to configure
It'll be great to have
- a single configuration file that configures everything related to scheduling
- a single file with plugin points for individual schedulers
- coupled with a single parser with custom parse-points for individual
schedulers
- configuration file follow a hierarchical schema.
Note that none of this is new. MAPREDUCE-861 originally did this in Hadoop-1,
but it got lost during the rewrite that is YARN. This JIRA proposes to bring
all of it back.
> Support hierarchical and unified scheduler configuration
> --------------------------------------------------------
>
> Key: YARN-2986
> URL: https://issues.apache.org/jira/browse/YARN-2986
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
>
> Today's scheduler configuration is fragmented and non-intuitive, and needs to
> be improved. Details in comments.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
