[
https://issues.apache.org/jira/browse/YARN-140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ahmed Radwan updated YARN-140:
------------------------------
Attachment: YARN-140_rev4.patch
The cause of these failures seems to be a couple of missing default
configurations that were previously loaded from capacity-scheduler.xml. I have
added such defaults to CapacitySchedulerConfiguration.java. I have ran all
these failing tests locally, and all of them succeed except two
(TestApplicationCleanup & TestGetGroups) which fail with and without the patch,
so they seem to be unrelated to this patch.
The two additional defaults that I needed to add to fix these tests are:
1- To return the queue "root.default" as a child for root if no children are
defined for root.
2- To return a capacity of 100.0f for the queue "root.default" if it exists but
no capacity is specified for it. (I was unsure about this default considering
the case when root has other children specified, but I think it should be fine
since both undefined capacity (i.e., -1) or maximum capacity (i.e., 100) will
represent an invalid configuration.
I am attaching the updated patch.
> Add capacity-scheduler-default.xml to provide a default set of configurations
> for the capacity scheduler.
> ---------------------------------------------------------------------------------------------------------
>
> Key: YARN-140
> URL: https://issues.apache.org/jira/browse/YARN-140
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacityscheduler
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Attachments: YARN-140.patch, YARN-140_rev2.patch,
> YARN-140_rev3.patch, YARN-140_rev4.patch
>
>
> When setting up the capacity scheduler users are faced with problems like:
> {code}
> FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
> starting ResourceManager
> java.lang.IllegalArgumentException: Illegal capacity of -1 for queue root
> {code}
> Which basically arises from missing basic configurations, which in many
> cases, there is no need to explicitly provide, and a default configuration
> will be sufficient. For example, to address the error above, the user need to
> add a capacity of 100 to the root queue.
> So, we need to add a capacity-scheduler-default.xml, this will be helpful to
> provide the basic set of default configurations required to run the capacity
> scheduler. The user can still override existing configurations or provide new
> ones in capacity-scheduler.xml. This is similar to *-default.xml vs
> *-site.xml for yarn, core, mapred, hdfs, etc.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
