[ http://issues.apache.org/jira/browse/HADOOP-808?page=all ]
Arun C Murthy updated HADOOP-808: --------------------------------- Attachment: HADOOP-808_1_20061208.patch myjob.tgz Here is a simple fix to JobClient.java It handles the above deficiency by 'mapred.jar' in the jobconf, loads the 'mapred.job.class' (this class is assumed to extend org.apache.hadoop.util.ToolBase') and calls it's 'doMain'; there-by providing a hook for user code before the job is fired. Hence with this one can completey 'configure' the map-reduce job (job-name, key/value classes, output-directory, speculative-execution, no. of maps/reduces etc.), specify the 'jar' with the mapper/reducer classes and fire the job. Thus any change needed for 'configuration' is now done completely via the job.xml, and shouldn't need be done via JobConf.set* methods. With the attached myjob.jar/myjob.xml one can: $ hadoop job -submit myjob.xml to run a modified version of 'random-writer'... there-by illustrating it's usefulness. > Enhance 'bin/hadoop job -submit <>' > ----------------------------------- > > Key: HADOOP-808 > URL: http://issues.apache.org/jira/browse/HADOOP-808 > Project: Hadoop > Issue Type: Improvement > Components: mapred > Reporter: Arun C Murthy > Assigned To: Arun C Murthy > Priority: Minor > Attachments: HADOOP-808_1_20061208.patch, myjob.tgz > > > Today 'bin/hadoop job -submit' takes a config file and just does a > JobClient.submitJob(jobConf), this isn't flexible enough for cases where the > user wants to submit a job which calls his main(), does some work and then > calls 'JobClient.runJob', I propose we enhance this subcommand to handle that > scenario. This could be really useful for the time we decide to setup a > simple webpage with 'form' where the user uploads a job.xml & job.jar and we > can fire the job. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira