Hi Harsh, There is only three warnnings in stderr *stderr logs*
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapred.Child). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. On Fri, Jan 3, 2014 at 1:59 PM, Harsh J <ha...@cloudera.com> wrote: > Hi again, > > I did not mention the TaskTracker log, but the actual Task JVM's > syslog/stderr/stdout. Do those log *any* problems at all? > > On Fri, Jan 3, 2014 at 11:16 AM, Azuryy Yu <azury...@gmail.com> wrote: > > Hi Harsh, > > Thanks. > > > > There is no any error logs for attempt_201312201200_34795_r_000000_0 in > the > > tasktracker log. only '0.0% reduce > copy >' > > > > I configured all hosts in all slaves and master. > > > > This job has only one reduce. it hanged. but I configured everybody's max > > job running to '1' in the Fair scheduler file. > > > > but some people's max job running greater than one. and these people's > job > > never hanged... > > > > > > On Fri, Jan 3, 2014 at 1:13 PM, Harsh J <ha...@cloudera.com> wrote: > >> > >> Does the Reduce task log (of attempt_201312201200_34795_r_000000_0) > >> show any errors in trying to communicate with the various TaskTrackers > >> in trying to obtain the data? > >> > >> On Fri, Jan 3, 2014 at 9:54 AM, Azuryy Yu <azury...@gmail.com> wrote: > >> > Add addtional: > >> > > >> > Our MR version is 1.2.1, not 1.0.4 > >> > > >> > There is no useful information in the JT log. > >> > > >> > > >> > On Fri, Jan 3, 2014 at 12:20 PM, Azuryy Yu <azury...@gmail.com> > wrote: > >> >> > >> >> Hi, > >> >> > >> >> Our prod cluster met some issues recently, > >> >> All map tasks finished successfully, but reduce task hanged. > >> >> > >> >> but It's not happened on all TaskTrackers, only sometimes. we used > >> >> mapred-1.0.4 > >> >> > >> >> There is "0.0% reduce > copy >" forever until kill task manually. > >> >> > >> >> reduce logs on the TaskTracker: > >> >> hadoop-hadoop-tasktracker-10-200-91-186.out:14/01/03 06:13:57 INFO > >> >> mapred.TaskTracker: JVM with ID: jvm_201312201200_34795_r_-365330778 > >> >> given > >> >> task: attempt_201312201200_34795_r_000000_0 > >> >> hadoop-hadoop-tasktracker-10-200-91-186.out:14/01/03 06:14:04 INFO > >> >> mapred.TaskTracker: attempt_201312201200_34795_r_000000_0 0.0% > reduce > > >> >> copy > >> >> > > >> >> hadoop-hadoop-tasktracker-10-200-91-186.out:14/01/03 06:14:08 INFO > >> >> mapred.TaskTracker: attempt_201312201200_34795_r_000000_0 0.0% > reduce > > >> >> copy > >> >> > > >> >> hadoop-hadoop-tasktracker-10-200-91-186.out:14/01/03 06:14:14 INFO > >> >> mapred.TaskTracker: attempt_201312201200_34795_r_000000_0 0.0% > reduce > > >> >> copy > >> >> > > >> >> hadoop-hadoop-tasktracker-10-200-91-186.out:14/01/03 06:14:17 INFO > >> >> mapred.TaskTracker: attempt_201312201200_34795_r_000000_0 0.0% > reduce > > >> >> copy > >> >> > > >> >> hadoop-hadoop-tasktracker-10-200-91-186.out:14/01/03 06:14:23 INFO > >> >> mapred.TaskTracker: attempt_201312201200_34795_r_000000_0 0.0% > reduce > > >> >> copy > >> >> > > >> > > >> > > >> > >> > >> > >> -- > >> Harsh J > > > > > > > > -- > Harsh J >
