Does the Reduce task log (of attempt_201312201200_34795_r_000000_0) show any errors in trying to communicate with the various TaskTrackers in trying to obtain the data?
On Fri, Jan 3, 2014 at 9:54 AM, Azuryy Yu <[email protected]> wrote: > Add addtional: > > Our MR version is 1.2.1, not 1.0.4 > > There is no useful information in the JT log. > > > On Fri, Jan 3, 2014 at 12:20 PM, Azuryy Yu <[email protected]> wrote: >> >> Hi, >> >> Our prod cluster met some issues recently, >> All map tasks finished successfully, but reduce task hanged. >> >> but It's not happened on all TaskTrackers, only sometimes. we used >> mapred-1.0.4 >> >> There is "0.0% reduce > copy >" forever until kill task manually. >> >> reduce logs on the TaskTracker: >> hadoop-hadoop-tasktracker-10-200-91-186.out:14/01/03 06:13:57 INFO >> mapred.TaskTracker: JVM with ID: jvm_201312201200_34795_r_-365330778 given >> task: attempt_201312201200_34795_r_000000_0 >> hadoop-hadoop-tasktracker-10-200-91-186.out:14/01/03 06:14:04 INFO >> mapred.TaskTracker: attempt_201312201200_34795_r_000000_0 0.0% reduce > copy >> > >> hadoop-hadoop-tasktracker-10-200-91-186.out:14/01/03 06:14:08 INFO >> mapred.TaskTracker: attempt_201312201200_34795_r_000000_0 0.0% reduce > copy >> > >> hadoop-hadoop-tasktracker-10-200-91-186.out:14/01/03 06:14:14 INFO >> mapred.TaskTracker: attempt_201312201200_34795_r_000000_0 0.0% reduce > copy >> > >> hadoop-hadoop-tasktracker-10-200-91-186.out:14/01/03 06:14:17 INFO >> mapred.TaskTracker: attempt_201312201200_34795_r_000000_0 0.0% reduce > copy >> > >> hadoop-hadoop-tasktracker-10-200-91-186.out:14/01/03 06:14:23 INFO >> mapred.TaskTracker: attempt_201312201200_34795_r_000000_0 0.0% reduce > copy >> > > > -- Harsh J
