Thanks for the responses! -SM
On Tue, Sep 23, 2008 at 8:11 PM, Owen O'Malley <[EMAIL PROTECTED]> wrote: > On Tue, Sep 23, 2008 at 5:40 PM, Sandy <[EMAIL PROTECTED]> wrote: > > > > > I just wrote a combiner class to try and speed things up. However, now I > > want to do something like the following: > > ==map phase== > > input: key = LongWritable value = Text, > > output: key = Text, value = Longwritable > > > > ==combiner== > > input: key = Text, value = iterator<LongWritable> > > output: key = Text, value = Text > > > The input and output types for the combiner *must* be the same. The > combiner > may be applied 0, 1, or many times between the map and the reduce. So, > combiners must be: > * not depend on being run exactly once > * not have side effects > > InputFormat -> Map -> Combiner* -> Reduce -> OutputFormat > > Since the Combiner may run more than once, it can't do type > transformations. > > -- Owen >
