In pig 0.8 the default number of reducers changed from 1 to a value computed based on input data size - http://pig.apache.org/docs/r0.8.1/cookbook.html#Use+the+Parallel+Features
-Thejas On 5/27/11 6:46 AM, "Jonathan Coveney" <[email protected]> wrote: SET default_parallel X; will set the PARALLEL keyword for all parallel functions (ie set the reducers for the job) I am not sure how the default is calculated...for a while it was set to 1 I believe, ostensibly to force people to set it to something more reasonable. 2011/5/27 Harsh J <[email protected]> > The PARALLEL keyword controls the number of reducers used in the job. > If unspecified, a default number is applied. Is this what you're > looking for? > > On Fri, May 27, 2011 at 3:46 PM, Sudharsan Sampath > <[email protected]> wrote: > > > > Hi, > > > > Is there a reference on how the number of reducers required for a job is > calculated? > > > > Thanks > > Sudharsan S > > > > > > > > -- > Harsh J > --
