Thanks a lot for your explanation, Jothi.
So is this event generated by hadoop framework? Is there any API in mapper to
fire this event? Actually, I am thinking to implement a mapper that will emit
some <key, value> pairs, then fire this event to let the reducer works, the
same mapper task then emit some other <key, value> pairs and repeat. Do you
think is this logic feasible by current API?
Thanks,
Jianmin
________________________________
From: Jothi Padmanabhan <joth...@yahoo-inc.com>
To: core-user@hadoop.apache.org
Sent: Monday, June 1, 2009 12:26:31 PM
Subject: Re: question about when shuffle/sort start working
When a Mapper completes, MapCompletionEvents are generated. Reducers try to
fetch map outputs for a given map only on the receipt of such events.
Jothi
On 5/30/09 10:00 AM, "Jianmin Woo" <jianmin_...@yahoo.com> wrote:
> Hi,
> I am being confused by the protocol between mapper and reducer. When mapper
> emitting the (key,value) pair done, is there any signal the mapper send out to
> hadoop framework in protocol to indicate that map is done and the shuffle/sort
> can begin for reducer? If there is no this signal in protocol, when the
> framework begin the shuffle/sort?
>
> Thanks,
> Jianmin
>
>
>
>
