[
https://issues.apache.org/jira/browse/YARN-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14329917#comment-14329917
]
Wangda Tan commented on YARN-2986:
----------------------------------
An update:
After an offline discussion with [~vinodkv] and [~jianhe], now proposed
configuration file looks like:
{code}
<scheduler>
<type>capacity</type>
<maximum-applications>9999</maximum-applications>
<queue-mappings></queue-mappings>
<queue-mappings-override-enable></queue-mappings-override-enable>
<maximum-am-resource-percent>0.3</maximum-am-resource-percent>
<policy-properties>
<resource-calculator>
org.apache.hadoop.yarn.util.resource.DominantResourceCalculator
</resource-calculator>
</policy-properties>
<queue name="root">
<queues>
<queue name="default">
<state>RUNNING</state>
<acl_submit_applications>*</acl_submit_applications>
<acl_administer_queue>*</acl_administer_queue>
<accessible-node-labels>x</accessible-node-labels>
<policy-properties>
<user-limit-factor>2</user-limit-factor>
<capacity>50</capacity>
<maximum-capacity>90</maximum-capacity>
<node-locality-delay>30</node-locality-delay>
<node-labels>
<node-label name="x">
<capacity>20</capacity>
<maximum-capacity>50</maximum-capacity>
</node-label>
</node-labels>
</policy-properties>
</queue>
</queues>
</queue>
</scheduler>
{code}
One highlight of this proposal and previous proposal is: this contains a
"policy-properties" for each configuration node, which means a
scheduler-specific configurations, like capacity in CapacityScheduler and
minShare in FairScheduler, etc. ("policy" here means different kinds of
scheduling method).
For other common options (not belongs to a specific scheduler implementation),
should be placed outside of "policy-properties".
*Please feel free to share your thoughts about this proposal :).*
To move this forward, I filed several sub ticket, YARN-3233 is targeted to
solve the configuration file (for common scheduler and capacity scheduler)
definition and parsing, I will upload a patch right now. YARN-3234 is to solve
Capacity Scheduler integration with the new config file.
> (Umbrella) Support hierarchical and unified scheduler configuration
> -------------------------------------------------------------------
>
> Key: YARN-2986
> URL: https://issues.apache.org/jira/browse/YARN-2986
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Wangda Tan
> Attachments: YARN-2986.1.patch
>
>
> Today's scheduler configuration is fragmented and non-intuitive, and needs to
> be improved. Details in comments.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
