Here's a link to how one might want to specify number of reducers in pig: http://pig.apache.org/docs/r0.8.1/cookbook.html#Use+the+Parallel+Features
Though, at run time, how many reducer slots a job can get depends on other factors: job priority, job scheduler and total number of reducer slots in use at the same time that are allowed on a tasktracker. If there are some jobs running with higher priority, they will consume as many available reducer slots as possible. Lower priority jobs then have to either wait for availability of reducer resources and idle, or get smaller number of slots than configured... Hope this helps. Michael ________________________________ From: 唐亮 <[email protected]> To: [email protected] Sent: Thursday, September 15, 2011 12:47 AM Subject: Num Reducers is Very Small! Hi dear pigs: Sometimes when I run pig job with huge data, the number of reducers is very small(only 1 reducer), even if set PARALLEL, so the job runs extremely slow! Can I or how can I increase the number of reducers in pig job? Thanks!
