Could you please provide some small code snippets elaborating on how you implemented that? I have a similar need as the author of this thread and I would appreciate any help. Thanks!
Cheers, Sean Joman Chu-2 wrote: > > Hi, I use Toolrunner.run() for multiple MapReduce jobs. It seems to work > well. I've run sequences involving hundreds of MapReduce jobs in a for > loop and it hasn't died on me yet. > > On Wed, July 9, 2008 4:28 pm, Mori Bellamy said: >> Hey all, I'm trying to chain multiple mapreduce jobs together to >> accomplish a complex task. I believe that the way to do it is as follows: >> >> JobConf conf = new JobConf(getConf(), MyClass.class); //configure job.... >> set mappers, reducers, etc >> SequenceFileOutputFormat.setOutputPath(conf,myPath1); >> JobClient.runJob(conf); >> >> //new job JobConf conf2 = new JobConf(getConf(),MyClass.class) >> SequenceFileInputFormat.setInputPath(conf,myPath1); //more >> configuration... JobClient.runJob(conf2) >> >> Is this the canonical way to chain jobs? I'm having some trouble with >> this >> method -- for especially long jobs, the latter MR tasks sometimes do not >> start up. >> >> > > > -- > Joman Chu > AIM: ARcanUSNUMquam > IRC: irc.liquid-silver.net > > > -- View this message in context: http://www.nabble.com/How-to-chain-multiple-hadoop-jobs--tp18370089p18452309.html Sent from the Hadoop core-user mailing list archive at Nabble.com.
