Did you configure tez.lib.uris in tez-site.xml and put it under classpath ? It looks like you didn’t configure it.
Best Regard, Jeff Zhang From: amit kumar <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Thursday, June 25, 2015 at 11:58 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: Error while running Hive queries over tez No, I didn't add anything after first query, its like the hive query runs on fresh cluster and after that below error is shown for all queries. Below are the hive logs: 2015-06-24 05:12:50,163 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=task.TEZ.Stage-1 from=org.apache.hadoop.hive.ql.Driver> 2015-06-24 05:12:50,163 INFO [main]: ql.Driver (SessionState.java:printInfo(824)) - Launching Job 1 out of 1 2015-06-24 05:12:50,163 INFO [main]: ql.Driver (Driver.java:launchTask(1602)) - Starting task [Stage-1:MAPRED] in serial mode 2015-06-24 05:12:50,163 INFO [main]: tez.TezSessionPoolManager (TezSessionPoolManager.java:canWorkWithSameSession(217)) - The current user: amitkumar, session user: amitkumar 2015-06-24 05:12:50,163 INFO [main]: tez.TezSessionPoolManager (TezSessionPoolManager.java:canWorkWithSameSession(234)) - Current queue name is null incoming queue name is null 2015-06-24 05:12:50,190 INFO [main]: ql.Context (Context.java:getMRScratchDir(266)) - New scratch dir is hdfs://localhost:9000/tmp/hive/amitkumar/5dcd0855-107b-471e-8ae6-a332c8b7c919/hive_2015-06-24_05-12-50_013_722448017482877635-1 2015-06-24 05:12:50,204 INFO [main]: exec.Task (TezTask.java:updateSession(236)) - Session is already open 2015-06-24 05:12:50,207 INFO [main]: tez.DagUtils (DagUtils.java:localizeResource(951)) - Localizing resource because it does not exist: file:/home/amitkumar/Apache_tez/TEZ/tez-0.7.0 to dest: hdfs://localhost:9000/tmp/hive/amitkumar/_tez_session_dir/25e6f3e0-b913-46c8-9d87-83962f1dc073/tez-0.7.0 2015-06-24 05:12:50,212 INFO [main]: tez.DagUtils (DagUtils.java:localizeResource(955)) - Looks like another thread is writing the same file will wait. 2015-06-24 05:12:50,212 INFO [main]: tez.DagUtils (DagUtils.java:localizeResource(962)) - Number of wait attempts: 5. Wait interval: 5000 2015-06-24 05:13:15,239 ERROR [main]: tez.DagUtils (DagUtils.java:localizeResource(978)) - Could not find the jar that was being uploaded 2015-06-24 05:13:15,240 ERROR [main]: exec.Task (TezTask.java:execute(184)) - Failed to execute tez graph. java.io.IOException: Previous writer likely failed to write hdfs://localhost:9000/tmp/hive/amitkumar/_tez_session_dir/25e6f3e0-b913-46c8-9d87-83962f1dc073/tez-0.7.0. Failing because I am unlikely to write too. at org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeResource(DagUtils.java:979) at org.apache.hadoop.hive.ql.exec.tez.DagUtils.addTempResources(DagUtils.java:860) at org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeTempFilesFromConf(DagUtils.java:803) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.refreshLocalResourcesFromConf(TezSessionState.java:228) at org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:245) at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:136) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1604) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1364) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1177) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1004) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:994) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:201) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:153) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:364) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:712) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:631) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:570) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) 2015-06-24 05:13:15,240 ERROR [main]: ql.Driver (SessionState.java:printError(833)) - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask On Wed, Jun 24, 2015 at 2:23 PM, Jianfeng (Jeff) Zhang <[email protected]<mailto:[email protected]>> wrote: Did you add file/jar/archive after the first query ? It seems exception happens when you add resources. You can check the hive log for more info. By default hive.log is located in /tmp/${user}/hive.log Best Regard, Jeff Zhang From: amit kumar <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Wednesday, June 24, 2015 at 8:19 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Error while running Hive queries over tez Hi, I am running Hive with Tez. When I run Hive query below is the exception i am facing. 2015-06-24 05:13:15,240 ERROR [main]: exec.Task (TezTask.java:execute(184)) - Failed to execute tez graph. java.io.IOException: Previous writer likely failed to write hdfs://localhost:9000/tmp/hive/amitkumar/_tez_session_dir/25e6f3e0-b913-46c8-9d87-83962f1dc073/tez-0.7.0. Failing because I am unlikely to write too. at org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeResource(DagUtils.java:979) at org.apache.hadoop.hive.ql.exec.tez.DagUtils.addTempResources(DagUtils.java:860) at org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeTempFilesFromConf(DagUtils.java:803) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.refreshLocalResourcesFromConf(TezSessionState.java:228) at org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:245) at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:136) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1604) Please help me out what can be the issue. When i do fresh setup, Job runs only first time after that for each Hive query it shows same error. Thanks.
