When I remove MRHelpers.doJobClientMagic then NullPointerException in Configuration class occurs.
Could you advise me a base class (class and branch/release) for good practice in TEZ for mapReduce jobs? I've rewritten my MR job to use Counters (not available in MapReduce on TEZ) and Sessions (to improve iterative processing speed). I have just Map and Reduce phase, it works in loop (several iterations), so I think using session can improve a performance. Am I right? Kindly regards Wojciech Indyk 2014-05-21 0:33 GMT+02:00 Siddharth Seth <[email protected]>: > It's possible that the old Output Format is being used (mapred vs > mapreduce). > Could you try forcing this to use the new API with the following. > finalVertex.setBoolean("mapred.mapper.new-api", true); > Also, if you happen to be using MRHelpers.doJobClientMagic - remove that, > since that could reset this parameter. > > This is a little messed up, but we're working on making this much easier > to use in 0.5. > > Thanks > - Sid > > > > On Tue, May 20, 2014 at 3:19 PM, Wojciech Indyk > <[email protected]>wrote: > >> Hi all! >> I use tez-0.4 on HDP 2.1. I tried to save results of DAG as a >> SequenceFile. >> I use: >> finalVertex.set(MRJobConfig.OUTPUT_FORMAT_CLASS_ATTR, >> SequenceFileOutputFormat.class.getName()); >> The problem is the output is saved as TextOutputFormat. I use Sequence >> file as an input to DAG and it works fine (I use SequenceFileInputFormat). >> >> Kindly regards >> Wojciech Indyk >> > >
