MR launches multiple attempts for single Task in case of TaskAttempt failures or when speculative execution is turned on. In either case, a given Task will only ever have one successful TaskAttempt whose output will be accepted (committed).
Number of reduces is set to 1 by default in mapred-default.xml - you should explicitly set it to zero if you don't want reducers. By master, I suppose you mean JobTracker. JobTracker doesn't show all the attempts for a given Task, you should navigate to per-task page to see that. Thanks, +Vinod Kumar Vavilapalli Hortonworks Inc. http://hortonworks.com/ On Dec 9, 2012, at 6:53 AM, Oleg Zhurakousky wrote: > I studying user logs on the two node cluster that I have setup and I was > wondering if anyone can shed some light on these "attempt*' directories >> $ ls > attempt_201212051224_0021_m_000000_0 attempt_201212051224_0021_m_000003_0 > job-acls.xml > attempt_201212051224_0021_m_000002_0 attempt_201212051224_0021_r_000000_0 > > I mean its obvious that its talking about 3 attempts for Map task and 1 > attempt for reduce task. However my current MR job only results in some > output written to "attempt_201212051224_0021_m_000000_0". Nothing is the > reduce part (understandably since I don't even have a reducer, so my question > is: > > 1. The two more M attempts. . . what are they? > 2. Why was there an attempt to do a Reduce when no reducer was > provided.implemented > 3. Why my master node only had 1 attempt for M task but the slave had all > that's displayed and questioned above (the 'ls' output above is from the > slave node) > > Thanks > Oleg
signature.asc
Description: Message signed with OpenPGP using GPGMail
