You can extract the shuffle time from the job log. Take a look at
https://github.com/rajvish/hadoop-summary Raj >________________________________ > From: Bertrand Dechoux <decho...@gmail.com> >To: common-user@hadoop.apache.org >Sent: Monday, August 27, 2012 12:57 AM >Subject: Re: Measuring Shuffle time for MR job > >Shuffle time is considered as part of the reduce step. Without reduce, >there is no need for shuffling. >One way to measure it would be using the full reduce time with a >'/dev/null' reducer. > >I am not aware of any way to measure it. > >Regards > >Bertrand > >On Mon, Aug 27, 2012 at 8:18 AM, praveenesh kumar <praveen...@gmail.com>wrote: > >> Is there a way to know the total shuffle time of a map-reduce job - I mean >> some command or output that can tell that ? >> >> I want to measure total map, total shuffle and total reduce time for my MR >> job -- how can I achieve it ? I am using hadoop 0.20.205 >> >> >> Regards, >> Praveenesh >> > > > >-- >Bertrand Dechoux > > >
