[
https://issues.apache.org/jira/browse/HADOOP-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sreekanth Ramakrishnan updated HADOOP-4809:
-------------------------------------------
Attachment: HADOOP-4809-1.patch
Attaching patch which implements the same:
Following is the way the Feature is implemented:
1. Introduced a new Exception class called ConfigValidationException.
2. The get methods in CapacitySchedulerConf now throw an
ConfigValidationException when the values which are configured by the user are
invalid.
3. The methods to be validated by the configuration tool is annotated by
ValidatorDetail annotation which states if the particular method is dependent
on queue or not.
4. The config validator requires path of hadoop-conf-dir which contains both
hadoop-default.xml, hadoop-site.xml and capacity-scheduler.xml. The reason
hadoop-site.xml and hadoop-default.xml is required because configuration
validator requires to construct QueueManager object to find out number of
queues present in the system.
5. Using reflection Configuration validator finds out methods it requires to
validate. For queue dependent methods, it passes the queue name and for other
it simply makes a method call. If any exception is raised while executing the
exception message is printed.
6. Finally, the guaranteed capacity is checked cumulatively. If the gc exceeds
100% appropriate error is raised.
Added a script file config-validator in the capacity-scheduler/contrib/bin
folder. It is packaged into $HADOOP_HOME/contrib/capacity-scheduler
TestCase:
Test case consists of 4 tests:
1. Invalid GC testing.
2. Cumulative invalid GC testing.
The reason why GC alone is tested in first two test is because, if an invalid
GC is configured, the call to the method would be two times. One while you are
testing validity of queue dependent configuration and one for cumulative gc
testing. So I have split GC testing away from test number 3
3. Testing Invalid queue dependent values.
4. Testing Invalid queue independent values.
> Implement a config validator tool for the capacity scheduler
> ------------------------------------------------------------
>
> Key: HADOOP-4809
> URL: https://issues.apache.org/jira/browse/HADOOP-4809
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/capacity-sched
> Reporter: Hemanth Yamijala
> Assignee: Sreekanth Ramakrishnan
> Attachments: HADOOP-4809-1.patch
>
>
> The capacity scheduler sanity checks configuration when it starts and halts
> if there are any problems found. For ease of deployment, it would help to
> have a simple utility that will validate the configuration before the
> capacity scheduler can be started, and report errors / warnings to the user
> about (possible) misconfigurations.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
