[
https://issues.apache.org/jira/browse/HADOOP-785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12492807
]
Doug Cutting commented on HADOOP-785:
-------------------------------------
> I think it's also important not to try to overprotect users [ ... ]
I agree. Especially when doing so adds code that will need to be maintained.
I think marking things that should not normally be used as "Expert" should be
sufficient. The biggest problem with dangerous knobs in my experience is that
they generate spurious bug reports from folks who don't understand what they're
really for and abuse them. I think this is primarily a documentation problem.
We should advise reasonable values, and folks who ignore that and provide
ridiculous values deserve ridiculous behavior.
> Divide the server and client configurations
> -------------------------------------------
>
> Key: HADOOP-785
> URL: https://issues.apache.org/jira/browse/HADOOP-785
> Project: Hadoop
> Issue Type: Improvement
> Components: conf
> Affects Versions: 0.9.0
> Reporter: Owen O'Malley
> Assigned To: Milind Bhandarkar
>
> The configuration system is easy to misconfigure and I think we need to
> strongly divide the server from client configs.
> An example of the problem was a configuration where the task tracker has a
> hadoop-site.xml that set mapred.reduce.tasks to 1. Therefore, the job tracker
> had the right number of reduces, but the map task thought there was a single
> reduce. This lead to a hard to find diagnose failure.
> Therefore, I propose separating out the configuration types as:
> class Configuration;
> // reads site-default.xml, hadoop-default.xml
> class ServerConf extends Configuration;
> // reads hadoop-server.xml, $super
> class DfsServerConf extends ServerConf;
> // reads dfs-server.xml, $super
> class MapRedServerConf extends ServerConf;
> // reads mapred-server.xml, $super
> class ClientConf extends Configuration;
> // reads hadoop-client.xml, $super
> class JobConf extends ClientConf;
> // reads job.xml, $super
> Note in particular, that nothing corresponds to hadoop-site.xml, which
> overrides both client and server configs. Furthermore, the properties from
> the *-default.xml files should never be saved into the job.xml.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
