What is the number of reduce shuffle bytes for this job? Also, is this job CPU intensive on reducers or is it simple aggregation?
Sent from my iPhone On Mar 13, 2012, at 5:25 AM, Austin Chungath <[email protected]> wrote: > Hi, > I am running a pig query on around 500 GB input data. > The current block size is 128 MB and split size is the default 128 MB. > I have also specified 16 reducers and around 3800 mappers are running. > > Now I observe that shuffling is taking a long time to complete execution, > approximately 25 mins per job. > > Can anyone suggest how I can bring down the shuffling time? Is there any > property that I can tweak to improve performance? > > Thanks & Regards, > Austin
