Hi, I use Toolrunner.run() for multiple MapReduce jobs. It seems to work well. I've run sequences involving hundreds of MapReduce jobs in a for loop and it hasn't died on me yet.
On Wed, July 9, 2008 4:28 pm, Mori Bellamy said: > Hey all, I'm trying to chain multiple mapreduce jobs together to > accomplish a complex task. I believe that the way to do it is as follows: > > JobConf conf = new JobConf(getConf(), MyClass.class); //configure job.... > set mappers, reducers, etc > SequenceFileOutputFormat.setOutputPath(conf,myPath1); > JobClient.runJob(conf); > > //new job JobConf conf2 = new JobConf(getConf(),MyClass.class) > SequenceFileInputFormat.setInputPath(conf,myPath1); //more > configuration... JobClient.runJob(conf2) > > Is this the canonical way to chain jobs? I'm having some trouble with this > method -- for especially long jobs, the latter MR tasks sometimes do not > start up. > > -- Joman Chu AIM: ARcanUSNUMquam IRC: irc.liquid-silver.net
