[ http://issues.apache.org/jira/browse/HADOOP-167?page=all ]
Owen O'Malley updated HADOOP-167:
---------------------------------
Attachment: remove-confs.patch
This patch removes a lot of the extra JobConfs from the TaskTracker. In
particular, it does not read the config files for each map output as it is
transfered to a reduce. It also adds a new constructor for JobConf(Class) and
makes the old JobConf() constructor read the mapred-defaults.xml. Unless Nutch
makes heavy use of the JobConf(Configuration) and JobConf(Configuration,Class)
constructors, I think we should depriciate them. There really isn't any
advantage to having a separate Configuration object.
> reducing the number of Configuration & JobConf objects created
> --------------------------------------------------------------
>
> Key: HADOOP-167
> URL: http://issues.apache.org/jira/browse/HADOOP-167
> Project: Hadoop
> Type: Improvement
> Components: conf
> Versions: 0.1.1
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Fix For: 0.2
> Attachments: remove-confs.patch
>
> Currently, Configuration and JobConf objects are created many times during
> executing a job. In particular, the Task Tracker creates a lot of them. They
> both clutter up the logs and parse the xml config files over and over again.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
