Hi, Excuse me. May I ask a question about running Hive on Tez?
I've installed Hive on Tez, and running a simple query from hiveCli, hive> set hive.optimize.tez=true; hive> select * from table order by title_id limit 5; While, each time, I could see from the TezJobMonitor that, all the map jobs are done, but the reducer never get started, and the job is running forever there. I tried a number of times, and each time the same failure(job running hangs) happens again and again. Does anyone successfully running queries using Hive on Tez? Are there any tips or suggestions? Here is my job log: 13/12/13 20:57:31 INFO client.TezSession: Submitting dag to TezSession, sessionName=HIVE-365b35bc-2461-4e2f-83f9-8da1fa356a86, applicationId=application_1386896881353_0027 13/12/13 20:57:33 INFO client.TezSession: Submitted dag to TezSession, sessionName=HIVE-365b35bc-2461-4e2f-83f9-8da1fa356a86, applicationId=application_1386896881353_0027, dagId=dag_1386896881353_0027_1 13/12/13 20:57:33 INFO client.RMProxy: Connecting to ResourceManager at /10.183.195.180:9022 13/12/13 20:57:33 INFO log.PerfLogger: </PERFLOG method=TezSubmitDag start=1386968251250 end=1386968253338 duration=2088 from=org.apache.hadoop.hive.ql.exec.tez.TezTask> 13/12/13 20:57:33 INFO tez.TezJobMonitor: 13/12/13 20:57:33 INFO log.PerfLogger: <PERFLOG method=TezRunDag from=org.apache.hadoop.hive.ql.exec.tez.TezJobMonitor> 13/12/13 20:57:33 INFO log.PerfLogger: <PERFLOG method=TezSubmitToRunningDag from=org.apache.hadoop.hive.ql.exec.tez.TezJobMonitor> 13/12/13 20:57:33 INFO log.PerfLogger: </PERFLOG method=TezSubmitToRunningDag start=1386968253341 end=1386968253402 duration=61 from=org.apache.hadoop.hive.ql.exec.tez.TezJobMonitor> Status: Running (application id: application_1386896881353_0027) 13/12/13 20:57:33 INFO tez.TezJobMonitor: Status: Running (application id: application_1386896881353_0027) 13/12/13 20:57:33 INFO log.PerfLogger: <PERFLOG method=TezRunVertex.Reducer 2 from=org.apache.hadoop.hive.ql.exec.tez.TezJobMonitor> 13/12/13 20:57:33 INFO log.PerfLogger: <PERFLOG method=TezRunVertex.Map 1 from=org.apache.hadoop.hive.ql.exec.tez.TezJobMonitor> Map 1: -/- Reducer 2: -/- 13/12/13 20:57:33 INFO tez.TezJobMonitor: Map 1: -/- Reducer 2: -/- Map 1: -/- Reducer 2: 0/1 13/12/13 20:57:33 INFO tez.TezJobMonitor: Map 1: -/- Reducer 2: 0/1 Map 1: 0/16 Reducer 2: 0/1 13/12/13 20:57:34 INFO tez.TezJobMonitor: Map 1: 0/16 Reducer 2: 0/1 Map 1: 0/16 Reducer 2: 0/1 13/12/13 20:57:37 INFO tez.TezJobMonitor: Map 1: 0/16 Reducer 2: 0/1 Map 1: 0/16 Reducer 2: 0/1 13/12/13 20:57:40 INFO tez.TezJobMonitor: Map 1: 0/16 Reducer 2: 0/1 Map 1: 0/16 Reducer 2: 0/1 13/12/13 20:57:43 INFO tez.TezJobMonitor: Map 1: 0/16 Reducer 2: 0/1 Map 1: 0/16 Reducer 2: 0/1 13/12/13 20:57:46 INFO tez.TezJobMonitor: Map 1: 0/16 Reducer 2: 0/1 Map 1: 0/16 Reducer 2: 0/1 13/12/13 20:57:49 INFO tez.TezJobMonitor: Map 1: 0/16 Reducer 2: 0/1 Map 1: 0/16 Reducer 2: 0/1 13/12/13 20:57:52 INFO tez.TezJobMonitor: Map 1: 0/16 Reducer 2: 0/1 Map 1: 0/16 Reducer 2: 0/1 13/12/13 20:57:55 INFO tez.TezJobMonitor: Map 1: 0/16 Reducer 2: 0/1 Map 1: 1/16 Reducer 2: 0/1 13/12/13 20:57:56 INFO tez.TezJobMonitor: Map 1: 1/16 Reducer 2: 0/1 Map 1: 2/16 Reducer 2: 0/1 13/12/13 20:57:58 INFO tez.TezJobMonitor: Map 1: 2/16 Reducer 2: 0/1 Map 1: 3/16 Reducer 2: 0/1 13/12/13 20:57:58 INFO tez.TezJobMonitor: Map 1: 3/16 Reducer 2: 0/1 Map 1: 5/16 Reducer 2: 0/1 13/12/13 20:57:59 INFO tez.TezJobMonitor: Map 1: 5/16 Reducer 2: 0/1 Map 1: 8/16 Reducer 2: 0/1 13/12/13 20:57:59 INFO tez.TezJobMonitor: Map 1: 8/16 Reducer 2: 0/1 Map 1: 12/16 Reducer 2: 0/1 13/12/13 20:57:59 INFO tez.TezJobMonitor: Map 1: 12/16 Reducer 2: 0/1 Map 1: 15/16 Reducer 2: 0/1 13/12/13 20:58:00 INFO tez.TezJobMonitor: Map 1: 15/16 Reducer 2: 0/1 13/12/13 20:58:00 INFO log.PerfLogger: </PERFLOG method=TezRunVertex.Map 1 start=1386968253402 end=1386968280223 duration=26821 from=org.apache.hadoop.hive.ql.exec.tez.TezJobMonitor> Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:00 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:03 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:06 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:09 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:12 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:15 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:18 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:21 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:24 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:27 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:30 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:33 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:36 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:39 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:42 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:45 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:48 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:51 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:54 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:58:57 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:00 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:03 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:06 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:09 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:12 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:15 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:18 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:21 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:24 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:27 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:30 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:33 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:36 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:39 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:42 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:45 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:48 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:52 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:55 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 20:59:58 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 21:00:01 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 21:00:04 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 21:00:07 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 21:00:10 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Map 1: 16/16 Reducer 2: 0/1 13/12/13 21:00:13 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1 Thanks, Zhenxiao
