[ http://issues.apache.org/jira/browse/HADOOP-167?page=comments#action_12376520 ]
Doug Cutting commented on HADOOP-167: ------------------------------------- Can we stop the extra reads caused by addFinalResource() and 'new JobConf(Configuration)' by re-using the hash table instead of re-reading the files? addFinalResource could simply read that single file, rather than re-read everything. And 'new JobConf(Configuration)' could clone the contents of the configuration rather than re-reading it, no? Or even use nested Properties... One feature that's currently supported is that Configuration.write() only writes things that differ from the defaults. This isn't essential, but it's nice. The way it distinguishes is that defaults are always in a nested properties and non-defaults are always in the top-level properties. > reducing the number of Configuration & JobConf objects created > -------------------------------------------------------------- > > Key: HADOOP-167 > URL: http://issues.apache.org/jira/browse/HADOOP-167 > Project: Hadoop > Type: Improvement > Components: conf > Versions: 0.1.1 > Reporter: Owen O'Malley > Assignee: Owen O'Malley > Fix For: 0.2 > Attachments: remove-confs.patch > > Currently, Configuration and JobConf objects are created many times during > executing a job. In particular, the Task Tracker creates a lot of them. They > both clutter up the logs and parse the xml config files over and over again. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
