As others have pointed out, its mostly applied as an optimization step. In most cases one's 'Mapper' outputs carry at least a small group of similar keys that go on to the reducer after a copy and a sort phase. To reduce it locally (in-memory) via a 'Combiner' helps reduce data in the copy-sort stages until the 'Reducer' operation kicks-in.
Do note that, implementation-wise, a 'combiner' class must always collect the same key-value pair types as the mapper function. On Mon, Aug 2, 2010 at 9:09 PM, Jackob Carlsson <[email protected]> wrote: > Hi everyone, > Could anyone please help me to understand the function of combiner? > > Thanks in advance > Jackob > -- Harsh J www.harshj.com
