Which release of Hadoop are you running? What do the logs on the TaskTracker tell you during the time the slow tasks are getting launched?
hadoop-0.20.203 has a ton of bug fixes since hadoop-0.20.2 which help fix issues with slow launches - you might want to upgrade. Arun On Aug 5, 2011, at 11:02 AM, Iman E wrote: > Hello all, > I have a question regarding the mappers. I can see from the logs that the > start time of the mapper is different from start time of logging. I am having > a problem because that time difference sometimes is few seconds, but other > times it is > > For example, one mapper that is supposed to read 65 MB. Its start time is > 12:30:53 whereis the logging start time is 12:33:01 and the end time is > 12:33:20. All the laoded data are local to the same rack. > In a perfect run, these numbers are as follows: the start time is 18:15:45, > logging start time: 18:15:48, and end time: 18:16:02. > > > I am running a job of more than 2400 mapper. Because of the above problem, > instead of the job taking 15-20 mins on 14 machine ( it happened in few > runs), other times it is taking more than 70 mins. Any suggestions how to fix > this problem or what could possibly be causing it. > > Thanks, > Iman
