On Thu, 18 Aug 2011 13:44:22 -0700, vipul sharma <sharmavipulw...@gmail.com> wrote: > *I think the error is due to using combiner. Since combiner is output data > in Text and Reducer is expecting IntArrayWritable. If I remove combiner > everything works. What am I doing wrong and how can I get the combiner to > work? Any help is greatly appreciated.
It's not immediately apparent, but combiners may run "tiered". That is, the mapper output may be run through a bunch of combiners, and THEIR output may be run through another tier of combiners, and so on. Thus the combiner input/output types must be exactly the same. You'll have to write another Combiner class, which handles the combining, and then have your Reducer class do the combining AND the conversion into output values. In the absence of chainreducers, I'd suggest creating an abstract MyCombinerReducer class which implements the combining algorithm, then derive both the Combiner and Reducer from it so they can both call the same combining code.
