[ http://issues.apache.org/jira/browse/HADOOP-808?page=all ]

Arun C Murthy updated HADOOP-808:
---------------------------------

    Attachment: HADOOP-808_1_20061208.patch
                myjob.tgz

Here is a simple fix to JobClient.java

It handles the above deficiency by 'mapred.jar' in the jobconf, loads the 
'mapred.job.class' (this class is assumed to extend 
org.apache.hadoop.util.ToolBase') and calls it's 'doMain'; there-by providing a 
hook for user code before the job is fired.

Hence with this one can completey 'configure' the map-reduce job (job-name, 
key/value classes, output-directory, speculative-execution, no. of maps/reduces 
etc.), specify the 'jar' with the mapper/reducer classes and fire the job. Thus 
any change needed for 'configuration' is now done completely via the job.xml, 
and shouldn't need be done via JobConf.set* methods.

With the attached myjob.jar/myjob.xml one can:
$ hadoop job -submit myjob.xml
to run a modified version of 'random-writer'... there-by illustrating it's 
usefulness.

> Enhance 'bin/hadoop job -submit <>'
> -----------------------------------
>
>                 Key: HADOOP-808
>                 URL: http://issues.apache.org/jira/browse/HADOOP-808
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Arun C Murthy
>         Assigned To: Arun C Murthy
>            Priority: Minor
>         Attachments: HADOOP-808_1_20061208.patch, myjob.tgz
>
>
> Today 'bin/hadoop job -submit' takes a config file and just does a 
> JobClient.submitJob(jobConf), this isn't flexible enough for cases where the 
> user wants to submit a job which calls his main(), does some work and then 
> calls 'JobClient.runJob', I propose we enhance this subcommand to handle that 
> scenario. This could be really useful for the time we decide to setup a 
> simple webpage with 'form' where the user uploads a job.xml & job.jar and we 
> can fire the job.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: 
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Reply via email to