I don't mean the # of reducers of a job is changed from its configuration, but any point of time the number of reducer tasks it can run is related to those factors I mentioned b4.
Michael ________________________________ From: Norbert Burger <[email protected]> To: [email protected]; jiang licht <[email protected]> Sent: Thursday, September 15, 2011 4:35 PM Subject: Re: Num Reducers is Very Small! On Thu, Sep 15, 2011 at 1:16 PM, jiang licht <[email protected]> wrote: > Lower priority jobs then have to either wait for availability of reducer > resources and idle, or get smaller number of slots than configured... This last part ("or get smaller number...") isn't right, is it? The number of reducer tasks is a function of the Pig script design. While any particular job can be forced to wait until it receives enough slots to complete processing, the cluster state shouldn't force a change in script configuration, right? Norbert
