On 11/24/09 12:55 PM, "z3r0c001" <[email protected]> wrote: > this is a strange problem so please bear with me, I submitted about 290 > jobs on my queue on a 6 machine cluster (15 map slots, 15 reduce slots) , > and everything was fine and dandy for first 15 hours, I noticed that some > jobs started reduce tasks even before single map had been executed, so out > of 15 reduce slots, 12 were taken up by one inactive job, with status as > reduce -> copy, but no progress, so one active job had only 3 slots to run > with, and that slowed up overall execution, > > please fell free to tell me if that was incoherent, but is there a way you > can tell queue to not execute reduce before map is done, I've speculative > execution flag as FALSE.
mapred.reduce.slowstart.completed.maps is the setting you are looking for. See also https://issues.apache.org/jira/browse/MAPREDUCE-1184
