[ http://issues.apache.org/jira/browse/HADOOP-785?page=comments#action_12456562 ] Doug Cutting commented on HADOOP-785: -------------------------------------
> should we think about about separating the 'ClientConfiguration' into static > and dynamic parts? To some degree we already have that: the static part is in the config file and the dynamic part is the object. We support specifying properties on the command line, so that one can run things like: bin/hadoop jar -Dlog.level=DEBUG ... I think this works with most commands since HADOOP-59 was committed. > Divide the server and client configurations > ------------------------------------------- > > Key: HADOOP-785 > URL: http://issues.apache.org/jira/browse/HADOOP-785 > Project: Hadoop > Issue Type: Improvement > Components: conf > Affects Versions: 0.9.0 > Reporter: Owen O'Malley > Assigned To: Milind Bhandarkar > Fix For: 0.10.0 > > > The configuration system is easy to misconfigure and I think we need to > strongly divide the server from client configs. > An example of the problem was a configuration where the task tracker has a > hadoop-site.xml that set mapred.reduce.tasks to 1. Therefore, the job tracker > had the right number of reduces, but the map task thought there was a single > reduce. This lead to a hard to find diagnose failure. > Therefore, I propose separating out the configuration types as: > class Configuration; > // reads site-default.xml, hadoop-default.xml > class ServerConf extends Configuration; > // reads hadoop-server.xml, $super > class DfsServerConf extends ServerConf; > // reads dfs-server.xml, $super > class MapRedServerConf extends ServerConf; > // reads mapred-server.xml, $super > class ClientConf extends Configuration; > // reads hadoop-client.xml, $super > class JobConf extends ClientConf; > // reads job.xml, $super > Note in particular, that nothing corresponds to hadoop-site.xml, which > overrides both client and server configs. Furthermore, the properties from > the *-default.xml files should never be saved into the job.xml. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
