[ http://issues.apache.org/jira/browse/HADOOP-785?page=comments#action_12456546 ] Arun C Murthy commented on HADOOP-785: --------------------------------------
+1 Another view-point: One of the things I frequently wish I had goes something like this - for a particular job/task I'd like to tweak the log-level to 'debug' while developing/testing... Generalising: while we are at it, should we think about about separating the 'ClientConfiguration' into static and dynamic parts? The 'dynamic' aspect would cover cases like: switch log-level, turn speculative execution on/off etc. I concede it might be more useful for developers rather than users in the short/medium term... Thoughts? Does it sound like a good direction? > Divide the server and client configurations > ------------------------------------------- > > Key: HADOOP-785 > URL: http://issues.apache.org/jira/browse/HADOOP-785 > Project: Hadoop > Issue Type: Improvement > Components: conf > Affects Versions: 0.9.0 > Reporter: Owen O'Malley > Assigned To: Milind Bhandarkar > Fix For: 0.10.0 > > > The configuration system is easy to misconfigure and I think we need to > strongly divide the server from client configs. > An example of the problem was a configuration where the task tracker has a > hadoop-site.xml that set mapred.reduce.tasks to 1. Therefore, the job tracker > had the right number of reduces, but the map task thought there was a single > reduce. This lead to a hard to find diagnose failure. > Therefore, I propose separating out the configuration types as: > class Configuration; > // reads site-default.xml, hadoop-default.xml > class ServerConf extends Configuration; > // reads hadoop-server.xml, $super > class DfsServerConf extends ServerConf; > // reads dfs-server.xml, $super > class MapRedServerConf extends ServerConf; > // reads mapred-server.xml, $super > class ClientConf extends Configuration; > // reads hadoop-client.xml, $super > class JobConf extends ClientConf; > // reads job.xml, $super > Note in particular, that nothing corresponds to hadoop-site.xml, which > overrides both client and server configs. Furthermore, the properties from > the *-default.xml files should never be saved into the job.xml. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
