[ https://issues.apache.org/jira/browse/YARN-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14329917#comment-14329917 ]
Wangda Tan commented on YARN-2986: ---------------------------------- An update: After an offline discussion with [~vinodkv] and [~jianhe], now proposed configuration file looks like: {code} <scheduler> <type>capacity</type> <maximum-applications>9999</maximum-applications> <queue-mappings></queue-mappings> <queue-mappings-override-enable></queue-mappings-override-enable> <maximum-am-resource-percent>0.3</maximum-am-resource-percent> <policy-properties> <resource-calculator> org.apache.hadoop.yarn.util.resource.DominantResourceCalculator </resource-calculator> </policy-properties> <queue name="root"> <queues> <queue name="default"> <state>RUNNING</state> <acl_submit_applications>*</acl_submit_applications> <acl_administer_queue>*</acl_administer_queue> <accessible-node-labels>x</accessible-node-labels> <policy-properties> <user-limit-factor>2</user-limit-factor> <capacity>50</capacity> <maximum-capacity>90</maximum-capacity> <node-locality-delay>30</node-locality-delay> <node-labels> <node-label name="x"> <capacity>20</capacity> <maximum-capacity>50</maximum-capacity> </node-label> </node-labels> </policy-properties> </queue> </queues> </queue> </scheduler> {code} One highlight of this proposal and previous proposal is: this contains a "policy-properties" for each configuration node, which means a scheduler-specific configurations, like capacity in CapacityScheduler and minShare in FairScheduler, etc. ("policy" here means different kinds of scheduling method). For other common options (not belongs to a specific scheduler implementation), should be placed outside of "policy-properties". *Please feel free to share your thoughts about this proposal :).* To move this forward, I filed several sub ticket, YARN-3233 is targeted to solve the configuration file (for common scheduler and capacity scheduler) definition and parsing, I will upload a patch right now. YARN-3234 is to solve Capacity Scheduler integration with the new config file. > (Umbrella) Support hierarchical and unified scheduler configuration > ------------------------------------------------------------------- > > Key: YARN-2986 > URL: https://issues.apache.org/jira/browse/YARN-2986 > Project: Hadoop YARN > Issue Type: Improvement > Reporter: Vinod Kumar Vavilapalli > Assignee: Wangda Tan > Attachments: YARN-2986.1.patch > > > Today's scheduler configuration is fragmented and non-intuitive, and needs to > be improved. Details in comments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)