Wangda Tan commented on YARN-2986:

Thanks for [~vinodkv] reporting, a huge +1 for the proposal.

*In addition to Vinod's suggest, I think we should define scheduler conf 
standard first, my proposal is*


   <queue name="root">


      <queue name="a">

         <queue name="a1">

      <queue name="b">

There're several things need to be highlight in above example:
1) There's no "group xml node", instead of
        <queue name=1/>
        <queue name=2/>

    <!-- Scheduler specific global configuration -->
    <!-- For e.g.

I would suggest
<queue name=1/>
<queue name=2/>

2) Element vs. Attribute
(Definition of E/A, see http://www.w3schools.com/xml/xml_attributes.asp)
I think we shouldn't mix element/attribute in config file, attribute field 
should be only a limited set of properties. IMO, name/type should be only 
properties that need to put as attribute.
With this, admin will not hesitate about which property needs go to attribute 
and which needs to go Element.

*To Implement this, I think there're two steps:*

1) A parser to support easily get/set values for scheduler conf.
- Hide complexities of a generic XML parser
- Have basic functionalities to handle value inherit, etc.
For each level ({{<key>...</key>}}), data structure could as simple as:
SchedulerConfNode {
        String getName();
        String getType();
        String get(String key, bool inherit=false, default=null);
        List<Node> getChildren(String key);

2) Common implementation of existing Hadoop scheduler
Since there're some fields common in fair/capacity/fifo scheduler, we can 
extend the {{SchedulerConfNode}} to {{BaseSchedulerConfNode}}, which has more 
methods like:
BaseSchedulerConfNode : SchedulerConfNode {
        Resource getMinimumAllocation();
        Resource getMaximumAllocation();
        List<String> getSubmitAcls();
        // ...


> Support hierarchical and unified scheduler configuration
> --------------------------------------------------------
>                 Key: YARN-2986
>                 URL: https://issues.apache.org/jira/browse/YARN-2986
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
> Today's scheduler configuration is fragmented and non-intuitive, and needs to 
> be improved. Details in comments.

This message was sent by Atlassian JIRA

Reply via email to