I checke the Hive logs, below is the trace 2015-06-24 21:41:15,839 INFO [main]: exec.Utilities (Utilities.java:setBaseWork(670)) - Setting plan: /tmp/hive/amitkumar/5dcd0855-107b-471e-8ae6-a332c8b7c919/hive_2015-06-24_21-40-42_728_8986147354066924938-1/amitkumar/_tez_scratch_dir/582170f3-3abe-4c0d-99c6-0866cc7e6789/map.xml 2015-06-24 21:41:16,159 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=TezCreateVertex.Map 1 start=1435182075763 end=1435182076159 duration=396 from=org.apache.hadoop.hive.ql.exec.tez.TezTask> 2015-06-24 21:41:16,181 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=TezBuildDag start=1435182074938 end=1435182076181 duration=1243 from=org.apache.hadoop.hive.ql.exec.tez.TezTask> 2015-06-24 21:41:16,181 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=TezSubmitDag from=org.apache.hadoop.hive.ql.exec.tez.TezTask> 2015-06-24 21:41:16,224 INFO [main]: client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=HIVE-25e6f3e0-b913-46c8-9d87-83962f1dc073, applicationId=application_1435118621366_0002, dagName=amitkumar_20150624214040_41a00b43-061d-4571-b12f-614469ee5ea4:6 2015-06-24 21:41:16,646 INFO [main]: client.TezClientUtils (TezClientUtils.java:getSessionAMProxy(856)) - Application not running, applicationId=application_1435118621366_0002, yarnApplicationState=FINISHED, finalApplicationStatus=SUCCEEDED, trackingUrl=http://impetus-d901:8088/proxy/application_1435118621366_0002/A, diagnostics=Session stats:submittedDAGs=1, successfulDAGs=1, failedDAGs=0, killedDAGs=0
2015-06-24 21:41:16,674 INFO [main]: exec.Task (SessionState.java:printInfo(824)) - Tez session was closed. Reopening... 2015-06-24 21:41:16,674 INFO [main]: tez.TezSessionPoolManager (TezSessionPoolManager.java:close(174)) - Closing tez session default? false 2015-06-24 21:41:16,674 INFO [main]: tez.TezSessionState (TezSessionState.java:close(263)) - Closing Tez Session 2015-06-24 21:41:16,684 INFO [main]: client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=HIVE-25e6f3e0-b913-46c8-9d87-83962f1dc073, applicationId=application_1435118621366_0002 2015-06-24 21:41:16,712 INFO [main]: client.TezClientUtils (TezClientUtils.java:getSessionAMProxy(856)) - Application not running, applicationId=application_1435118621366_0002, yarnApplicationState=FINISHED, finalApplicationStatus=SUCCEEDED, trackingUrl=http://impetus-d901:8088/proxy/application_1435118621366_0002/A, diagnostics=Session stats:submittedDAGs=1, successfulDAGs=1, failedDAGs=0, killedDAGs=0 2015-06-24 21:41:16,712 INFO [main]: client.TezClient (TezClient.java:stop(509)) - Failed to shutdown Tez Session via proxy org.apache.tez.dag.api.SessionNotRunning: Application not running, applicationId=application_1435118621366_0002, yarnApplicationState=FINISHED, finalApplicationStatus=SUCCEEDED, trackingUrl=http://impetus-d901:8088/proxy/application_1435118621366_0002/A, diagnostics=Session stats:submittedDAGs=1, successfulDAGs=1, failedDAGs=0, killedDAGs=0 at org.apache.tez.client.TezClientUtils.getSessionAMProxy(TezClientUtils.java:857) at org.apache.tez.client.TezClient.getSessionAMProxy(TezClient.java:699) at org.apache.tez.client.TezClient.stop(TezClient.java:501) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.close(TezSessionState.java:265) at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.close(TezSessionPoolManager.java:176) at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.closeAndOpen(TezSessionPoolManager.java:282) at org.apache.hadoop.hive.ql.exec.tez.TezTask.submit(TezTask.java:381) at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:162) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1604) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1364) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1177) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1004) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:994) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:201) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:153) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:364) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:712) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:631) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:570) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) 2015-06-24 21:41:16,712 INFO [main]: client.TezClient (TezClient.java:stop(514)) - Could not connect to AM, killing session via YARN, sessionName=HIVE-25e6f3e0-b913-46c8-9d87-83962f1dc073, applicationId=application_1435118621366_0002 2015-06-24 21:41:18,457 INFO [main]: impl.YarnClientImpl (YarnClientImpl.java:killApplication(364)) - Killed application application_1435118621366_0002 2015-06-24 21:41:18,457 INFO [main]: tez.TezSessionState (TezSessionState.java:open(142)) - User of session id 25e6f3e0-b913-46c8-9d87-83962f1dc073 is amitkumar 2015-06-24 21:41:18,463 INFO [main]: tez.DagUtils (DagUtils.java:localizeResource(951)) - Localizing resource because it does not exist: file:/home/amitkumar/Apache_tez/TEZ/tez-0.7.0 to dest: hdfs://localhost:9000/tmp/hive/amitkumar/_tez_session_dir/25e6f3e0-b913-46c8-9d87-83962f1dc073/tez-0.7.0 2015-06-24 21:41:18,467 INFO [main]: tez.DagUtils (DagUtils.java:localizeResource(955)) - Looks like another thread is writing the same file will wait. 2015-06-24 21:41:18,467 INFO [main]: tez.DagUtils (DagUtils.java:localizeResource(962)) - Number of wait attempts: 5. Wait interval: 5000 2015-06-24 21:41:43,480 ERROR [main]: tez.DagUtils (DagUtils.java:localizeResource(978)) - Could not find the jar that was being uploaded 2015-06-24 21:41:43,481 ERROR [main]: exec.Task (TezTask.java:execute(184)) - Failed to execute tez graph. java.io.IOException: Previous writer likely failed to write hdfs://localhost:9000/tmp/hive/amitkumar/_tez_session_dir/25e6f3e0-b913-46c8-9d87-83962f1dc073/tez-0.7.0. Failing because I am unlikely to write too. at org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeResource(DagUtils.java:979) at org.apache.hadoop.hive.ql.exec.tez.DagUtils.addTempResources(DagUtils.java:860) I am not getting it, why session is getting closed. Thanks On Wed, Jun 24, 2015 at 2:23 PM, Jianfeng (Jeff) Zhang < [email protected]> wrote: > > Did you add file/jar/archive after the first query ? It seems exception > happens when you add resources. > You can check the hive log for more info. By default hive.log is located > in /tmp/${user}/hive.log > > > Best Regard, > Jeff Zhang > > > From: amit kumar <[email protected]> > Reply-To: "[email protected]" <[email protected]> > Date: Wednesday, June 24, 2015 at 8:19 PM > To: "[email protected]" <[email protected]> > Subject: Error while running Hive queries over tez > > Hi, > > I am running Hive with Tez. When I run Hive query below is the exception > i am facing. > > 2015-06-24 05:13:15,240 ERROR [main]: exec.Task > (TezTask.java:execute(184)) - Failed to execute tez graph. > java.io.IOException: Previous writer likely failed to write > hdfs://localhost:9000/tmp/hive/amitkumar/_tez_session_dir/25e6f3e0-b913-46c8-9d87-83962f1dc073/tez-0.7.0. > Failing because I am unlikely to write too. > at > org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeResource(DagUtils.java:979) > at > org.apache.hadoop.hive.ql.exec.tez.DagUtils.addTempResources(DagUtils.java:860) > at > org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeTempFilesFromConf(DagUtils.java:803) > at > org.apache.hadoop.hive.ql.exec.tez.TezSessionState.refreshLocalResourcesFromConf(TezSessionState.java:228) > at > org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:245) > at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:136) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1604) > > > Please help me out what can be the issue. When i do fresh setup, Job runs > only first time after that for each Hive query it shows same error. > > Thanks. > > >
