On Mon, Aug 2, 2010 at 4:28 PM, Jackob Carlsson <[email protected]> wrote: > Thanks Nick, but "in-memory" means a combiner can only be used over a single > mapper?right?! Is there a way we use it for several mappers as well? Also > what do you mean by "it may or may not run on a particular map attempt"? > > Br, > Jackob > > On Mon, Aug 2, 2010 at 5:43 PM, Nick Jones <[email protected]> wrote: > >> Hi Jackob, >> A combiner acts a lot like a reduce step but it's executed on the mapper >> with in-memory data. I've seen a reduction in job execution time by adding >> one. The one caveat to keep in mind is that it may or may not run on a >> particular map attempt. >> >> Nick >> >> >> >> On 8/2/2010 10:39 AM, Jackob Carlsson wrote: >> >>> Hi everyone, >>> Could anyone please help me to understand the function of combiner? >>> >>> Thanks in advance >>> Jackob >>> >>> >> >> >
> Is there a way we use it for several mappers as well? No. That is the exact opposite goal of the combiner. It runs locally. >it may or may not run on a particular map attempt It only runs when certain thresholds in the framework are reached. http://philippeadjiman.com/blog/2010/01/14/hadoop-tutorial-series-issue-4-to-use-or-not-to-use-a-combiner/
