A couple of things to do: 1) (optional) For tez.lib.uris, set it to "${fs.defaultFS}/apps/tez/tez.tar.gz” - this tez.tar.gz should come from tez-dist/target/. Given that your basic job is working, you can ignore this for now but it is the recommended way to deploy tez. 2) On a fresh setup, do a "hadoop dfs -mkdir -p /tmp/hive" and set permissions to 777 for both /tmp and /tmp/hive before running any job. You should probably check whether your setup is using a different dfs.umask that is causing write permissions later on.
thanks — Hitesh On Jun 24, 2015, at 9:20 PM, amit kumar <aks...@gmail.com> wrote: > Below is the tez-site.xml > > <configuration> > <property> > <name>tez.lib.uris</name> > > <value>${fs.defaultFS}/apps/tez-0.7.0,${fs.defaultFS}/apps/tez-0.7.0/lib/</value> > </property> > </configuration> > > I ran OrderWordCount job and that run successfully, only with hive queries i > am facing this error. > > Please let me know if i missing something. > > > Thanks. > > On Thu, Jun 25, 2015 at 4:04 AM, Jianfeng (Jeff) Zhang > <jzh...@hortonworks.com> wrote: > Did you configure tez.lib.uris in tez-site.xml and put it under classpath ? > It looks like you didn’t configure it. > > > Best Regard, > Jeff Zhang > > > From: amit kumar <aks...@gmail.com> > Reply-To: "user@tez.apache.org" <user@tez.apache.org> > Date: Thursday, June 25, 2015 at 11:58 AM > To: "user@tez.apache.org" <user@tez.apache.org> > Subject: Re: Error while running Hive queries over tez > > No, I didn't add anything after first query, its like the hive query runs on > fresh cluster and after that below error is shown for all queries. Below are > the hive logs: > > 2015-06-24 05:12:50,163 INFO [main]: log.PerfLogger > (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=task.TEZ.Stage-1 > from=org.apache.hadoop.hive.ql.Driver> > 2015-06-24 05:12:50,163 INFO [main]: ql.Driver > (SessionState.java:printInfo(824)) - Launching Job 1 out of 1 > 2015-06-24 05:12:50,163 INFO [main]: ql.Driver > (Driver.java:launchTask(1602)) - Starting task [Stage-1:MAPRED] in serial mode > 2015-06-24 05:12:50,163 INFO [main]: tez.TezSessionPoolManager > (TezSessionPoolManager.java:canWorkWithSameSession(217)) - The current user: > amitkumar, session user: amitkumar > 2015-06-24 05:12:50,163 INFO [main]: tez.TezSessionPoolManager > (TezSessionPoolManager.java:canWorkWithSameSession(234)) - Current queue name > is null incoming queue name is null > 2015-06-24 05:12:50,190 INFO [main]: ql.Context > (Context.java:getMRScratchDir(266)) - New scratch dir is > hdfs://localhost:9000/tmp/hive/amitkumar/5dcd0855-107b-471e-8ae6-a332c8b7c919/hive_2015-06-24_05-12-50_013_722448017482877635-1 > 2015-06-24 05:12:50,204 INFO [main]: exec.Task > (TezTask.java:updateSession(236)) - Session is already open > 2015-06-24 05:12:50,207 INFO [main]: tez.DagUtils > (DagUtils.java:localizeResource(951)) - Localizing resource because it does > not exist: file:/home/amitkumar/Apache_tez/TEZ/tez-0.7.0 to dest: > hdfs://localhost:9000/tmp/hive/amitkumar/_tez_session_dir/25e6f3e0-b913-46c8-9d87-83962f1dc073/tez-0.7.0 > 2015-06-24 05:12:50,212 INFO [main]: tez.DagUtils > (DagUtils.java:localizeResource(955)) - Looks like another thread is writing > the same file will wait. > 2015-06-24 05:12:50,212 INFO [main]: tez.DagUtils > (DagUtils.java:localizeResource(962)) - Number of wait attempts: 5. Wait > interval: 5000 > 2015-06-24 05:13:15,239 ERROR [main]: tez.DagUtils > (DagUtils.java:localizeResource(978)) - Could not find the jar that was being > uploaded > 2015-06-24 05:13:15,240 ERROR [main]: exec.Task (TezTask.java:execute(184)) - > Failed to execute tez graph. > java.io.IOException: Previous writer likely failed to write > hdfs://localhost:9000/tmp/hive/amitkumar/_tez_session_dir/25e6f3e0-b913-46c8-9d87-83962f1dc073/tez-0.7.0. > Failing because I am unlikely to write too. > at > org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeResource(DagUtils.java:979) > at > org.apache.hadoop.hive.ql.exec.tez.DagUtils.addTempResources(DagUtils.java:860) > at > org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeTempFilesFromConf(DagUtils.java:803) > at > org.apache.hadoop.hive.ql.exec.tez.TezSessionState.refreshLocalResourcesFromConf(TezSessionState.java:228) > at > org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:245) > at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:136) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1604) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1364) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1177) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1004) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:994) > at > org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:201) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:153) > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:364) > at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:712) > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:631) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:570) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:601) > at org.apache.hadoop.util.RunJar.run(RunJar.java:221) > at org.apache.hadoop.util.RunJar.main(RunJar.java:136) > 2015-06-24 05:13:15,240 ERROR [main]: ql.Driver > (SessionState.java:printError(833)) - FAILED: Execution Error, return code 1 > from org.apache.hadoop.hive.ql.exec.tez.TezTask > > > On Wed, Jun 24, 2015 at 2:23 PM, Jianfeng (Jeff) Zhang > <jzh...@hortonworks.com> wrote: > > Did you add file/jar/archive after the first query ? It seems exception > happens when you add resources. > You can check the hive log for more info. By default hive.log is located in > /tmp/${user}/hive.log > > > Best Regard, > Jeff Zhang > > > From: amit kumar <aks...@gmail.com> > Reply-To: "user@tez.apache.org" <user@tez.apache.org> > Date: Wednesday, June 24, 2015 at 8:19 PM > To: "user@tez.apache.org" <user@tez.apache.org> > Subject: Error while running Hive queries over tez > > Hi, > > I am running Hive with Tez. When I run Hive query below is the exception i am > facing. > > 2015-06-24 05:13:15,240 ERROR [main]: exec.Task (TezTask.java:execute(184)) - > Failed to execute tez graph. > java.io.IOException: Previous writer likely failed to write > hdfs://localhost:9000/tmp/hive/amitkumar/_tez_session_dir/25e6f3e0-b913-46c8-9d87-83962f1dc073/tez-0.7.0. > Failing because I am unlikely to write too. > at > org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeResource(DagUtils.java:979) > at > org.apache.hadoop.hive.ql.exec.tez.DagUtils.addTempResources(DagUtils.java:860) > at > org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeTempFilesFromConf(DagUtils.java:803) > at > org.apache.hadoop.hive.ql.exec.tez.TezSessionState.refreshLocalResourcesFromConf(TezSessionState.java:228) > at > org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:245) > at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:136) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1604) > > > Please help me out what can be the issue. When i do fresh setup, Job runs > only first time after that for each Hive query it shows same error. > > Thanks. > > > >