For how to control Pig's parallelism: http://pig.apache.org/docs/r0.9.1/perf.html#parallel
For your 'running' task count bug: https://issues.apache.org/jira/browse/MAPREDUCE-486 (Which would mean that only 5 may be 'running', while the rest may have already completed and were awaiting finalization of outputs. Not to be worried about - its more of a UI bug.) On 01-Dec-2011, at 6:36 AM, Keren Ouaknine wrote: > Pig script allocates 15 nodes to run a pig script (how to configure that?) > but what happens on a 10 nodes cluster only? > UI shows that 15 reducers are running, and at the same time max number of > reducers is 10 (configured one reducer per machine). > > > > Thanks, > Keren > > Kind % Complete Num Tasks Pending Running Complete Killed > Failed/Killed > Task Attempts > map 100.00% > > 1 0 0 1 0 0 / 0 > reduce 5.92% > > > 90 70 15 5 0 0 / 0 > > Running Map Tasks Running Reduce Tasks Total Submissions Nodes > Occupied Map Slots Occupied Reduce Slots Reserved Map Slots > Reserved Reduce Slots Map Task Capacity Reduce Task Capacity Avg. > Tasks/Node Blacklisted Nodes Graylisted Nodes Excluded Nodes > 0 0 49 10 0 0 0 0 10 10 > 2.00 0 0 0 > > > > > -- > Keren Ouaknine > Cell: +972 54 2565404 > Web: www.kereno.com > >
