[
https://issues.apache.org/jira/browse/YARN-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257310#comment-14257310
]
Vinod Kumar Vavilapalli commented on YARN-2986:
-----------------------------------------------
I propose the following
- A single unified configuration file called yarn-scheduler.xml
- If this file is not present, old configuration files will be applied by
individual schedulers. Otherwise, it is the only configuration file loaded.
[Backwards compatibility]
- Proposal for the schema of the unified file
{code:xml}
<scheduler-configuration>
<scheduler-name><!--values fifo, capacity-scheduler, fair-scheduler. We
should stop exposing class-names --> </scheduler-name>
<minimum-allocation-size>memory=1024,vcores=1 <!-- Compact representation of
resources --><Minimum-allocation-size>
<maximum-allocation-size>memory=8192,vcores=32 <!-- Compact representation of
resources --><maximum-allocation-size>
<scheduler-custom-configuration>
<!-- Scheduler specific global configuration -->
<!-- For e.g.
<maximum-applications>10000</maximum-applications>
<resource-calculator>DominantResourceCalculator</resource-calculator>
-->
</scheduler-custom-configuration>
<queue>
<name>root</name>
<state>running</state>
<acl-submit-app>*</acl-submit-app>
<acl-modify-app>admins</acl-modify-app>
<properties>
<property key="capacity" value="100" />
<property key="maxcapacity" value="100" />
</properties>
<children>
<names>default, a</names>
<queue>
<name>default</name>
<state>stopped</state>
<acl-submit-app>*</acl-submit-app>
<acl-modify-app>*</acl-modify-app>
<properties>
<property key="capacity" value="60" />
<property key="maxcapacity" value="60" />
</properties>
</queue>
<queue>
<name>a</name>
<state>running</state>
<acl-submit-app>alice,bob</acl-submit-app>
<acl-modify-app>mark</acl-modify-app>
<properties>
<property key="capacity" value="40" />
<property key="maxcapacity" value="50" />
</properties>
<children>
<names>childOfA</names>
<state>running</state>
<acl-submit-app>alice</acl-submit-app>
<acl-modify-app>mark</acl-modify-app>
<properties>
<property key="capacity" value="100" />
<property key="maxcapacity" value="100" />
</properties>
</children>
</queue>
</children>
</queue>
<scheduler-configuration>
{code}
To do:
- Preemption monitors
Thoughts? Folks focusing on fair-scheduler, will this scheme work?
> Support hierarchical and unified scheduler configuration
> --------------------------------------------------------
>
> Key: YARN-2986
> URL: https://issues.apache.org/jira/browse/YARN-2986
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
>
> Today's scheduler configuration is fragmented and non-intuitive, and needs to
> be improved. Details in comments.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
