[ http://issues.apache.org/jira/browse/HADOOP-808?page=comments#action_12458004 ] Arun C Murthy commented on HADOOP-808: --------------------------------------
Owen > This patch just enables the config to have a mapred.job.class that must > extend ToolBase that can manipulate the config before it is submitted. Absolutely. For e.g. a common use-case is to 'deduce' the no. of maps/reduces depending on the 'cluster size' which can't be done with % bin/hadoop job -submit <jobconf.xml> today. Doug > If someone's going to write a main() like method, why couldn't they simply > write a main() that submits the job? I don't see what this simplifies. I can see your point Doug, however this just slightly 'simplifies' the user-code which now doesn't have to bother with 'job submission'; thus, 'hopefully' easing the user's workload. More of a convenience... In essence it just enhances 'hadoop job -submit' by providing a 'hook' to manipulate job.xml before submission so that certain 'dynamic' behaviour (depending on things like 'cluster size') can be retained. I'm 'ok' debunking this if people don't think this adds an real value... > Enhance 'bin/hadoop job -submit <>' > ----------------------------------- > > Key: HADOOP-808 > URL: http://issues.apache.org/jira/browse/HADOOP-808 > Project: Hadoop > Issue Type: Improvement > Components: mapred > Affects Versions: 0.9.1 > Reporter: Arun C Murthy > Assigned To: Arun C Murthy > Priority: Minor > Fix For: 0.10.0 > > Attachments: HADOOP-808_1_20061208.patch, myjob.tgz > > > Today 'bin/hadoop job -submit' takes a config file and just does a > JobClient.submitJob(jobConf), this isn't flexible enough for cases where the > user wants to submit a job which calls his main(), does some work and then > calls 'JobClient.runJob', I propose we enhance this subcommand to handle that > scenario. This could be really useful for the time we decide to setup a > simple webpage with 'form' where the user uploads a job.xml & job.jar and we > can fire the job. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
