http://pig.apache.org/docs/r0.14.0/start.html#Running+the+Pig+Scripts+in+Mapreduce+Mode+or+Tez+Mode Do you have the directory containing tez-site.xml added to the pig classpath? That is most likely the cause for the error.
Regards, Rohini On Tue, Mar 17, 2015 at 4:10 PM, Hitesh Shah <[email protected]> wrote: > You have yarn configured with log aggregation disabled ( > yarn.log-aggregation-enable to false ). In this scenario, the logs are on > the NodeManager where the AM was launched. For this, find the log-dirs > property configured in yarn-site.xml and use that to find the logs for your > application. If this is a multi-node cluster, you will need to find logs > from all nodes in your cluster. YARN logs aggregation if enabled does this > automatically and stores all logs in HDFS. > > In any case, based on your earlier logs: > > > 2015-03-16 23:39:01,576 [PigTezLauncher-0] INFO > org.apache.tez.client.TezClient - Tez Client Version: [ component=tez-api, > version=0.5.2, revision=5d728786b681e549a6f33375991dcce97cbf79f3, > SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, > buildTIme=20141104-1809 ] > > > and > > > > > org.apache.tez.client.TezClientUtils - Using tez.lib.uris > > value from configuration: > > hdfs://localhost:9000/apps/tez-0.6.0-SNAPSHOT/tez-0.6.0.tar.gz > > I think the inconsistency of versions might be the reason why things are > failing. Please check the ResourceManager UI for the application in > question and see if the diagnostics message points to a version issue. > > thanks > — Hitesh > > > On Mar 17, 2015, at 3:51 PM, dagriq <[email protected]> wrote: > > > hi > > > > and thanks for the quick reply. > > I have tried the hint that you gave me but it was nothing written. > > Please open the attached screenshot; I issed the command after error was > raised... > > > > > > regards > > > > > > -------------------------------------------- > > On Tue, 3/17/15, Hitesh Shah <[email protected]> wrote: > > > > Subject: Re: error after installation - TezSession has already shutdown > > To: [email protected] > > Received: Tuesday, March 17, 2015, 11:30 AM > > > > Hello > > > > First issue from the stack > > trace: "org.apache.tez.dag.api.TezUncheckedException: > > Invalid configuration of tez jars, tez.lib.uris is not > > defined in the configuration”. It looks like your first > > run failed due to this. The second run seemed to be > > configured correctly but failed for a different reason. For > > this, look for a line along the lines of > > "org.apache.hadoop.yarn.client.api.impl.YarnClientImpl > > - Submitted application application_1426563447689_0001” to > > find the application ID. Now, invoke “bin/yarn logs > > -applicationId <appId>” to get all the logs and > > attach it to your email to this list so that we can take a > > look to see if there are other issues. > > > > Actually, my guess is that you have 2 different > > versions of tez in use. Based on the stack trace, you seem > > to be using tez-0.5.2 on the client machine but your tarball > > on HDFS is tez-0.6.0. Both of these need to have matching > > versions. If you go to the ResourceManager and look at the > > UI for the applicationId, its diagnostics might likely tell > > you that the AM was incompatible with the client. If you see > > a different error, please send across the logs from the > > above command and we can help further. > > > > thanks > > — HItesh > > > > > > > > > > > > > > > > > > > > On > > Mar 17, 2015, at 6:27 AM, dagriq <[email protected]> > > wrote: > > > >> hi > >> > >> I have downloaded > > and compiled Tez and I received one error when I tried at > > the bash shell: [ pig -x tez myscript.pig ]. > >> > >> I executed all steps > > from the [ http://tez.apache.org/install.html ] and > > I have a new folder in /usr/local/apache-tez. > >> I added one more subfolder called conf > > where I stored the new file tez-site.xml. > >> My first error message is copied below: > >> > >> > >> 2015-03-16 23:39:00,463 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler > > - For vertex - scope-34: parallelism=1, memory=1024, java > > opts=-Djava.net.preferIPv4Stack=true > > -Dhadoop.metrics.log.level=WARN -Xmx200m > > -Dlog4j.configuration=tez-container-log4j.properties > > -Dyarn.app.container.log.dir=<LOG_DIR> > > -Dtez.root.logger=INFO,CLA > >> 2015-03-16 > > 23:39:00,862 [PigTezLauncher-0] INFO > > org.apache.pig.tools.pigstats.tez.TezScriptState - Pig > > script settings are added to the job > >> > > 2015-03-16 23:39:01,576 [PigTezLauncher-0] INFO > > org.apache.tez.client.TezClient - Tez Client Version: [ > > component=tez-api, version=0.5.2, > > revision=5d728786b681e549a6f33375991dcce97cbf79f3, > > SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, > > buildTIme=20141104-1809 ] > >> > > 2015-03-16 23:39:01,757 [PigTezLauncher-0] INFO > > org.apache.hadoop.yarn.client.RMProxy - Connecting to > > ResourceManager at /0.0.0.0:8032 > >> > > 2015-03-16 23:39:02,798 [PigTezLauncher-0] INFO > > org.apache.tez.client.TezClient - Session mode. Starting > > session. > >> > >> > > 2015-03-16 23:39:02,800 [PigTezLauncher-0] ERROR > > org.apache.pig.backend.hadoop.executionengine.tez.TezJob - > > Cannot submit DAG > >> > >> > > org.apache.tez.dag.api.TezUncheckedException: Invalid > > configuration of tez jars, tez.lib.uris is not defined in > > the configuration > >> > >> at > > > org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:162) > >> at > > org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:711) > >> at > > org.apache.tez.client.TezClient.start(TezClient.java:294) > >> at > > > org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:95) > >> at > > > org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:195) > >> at > > > org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:159) > >> at > > > org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:167) > >> at > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > >> at > > java.util.concurrent.FutureTask.run(FutureTask.java:262) > >> at > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > >> at > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > >> at > > java.lang.Thread.run(Thread.java:745) > >> > > > >> 2015-03-16 23:39:03,758 [main] INFO > > org.apache.pig.tools.pigstats.tez.TezPigScriptStats - Script > > Statistics: > >> > >> > > HadoopVersion: 2.6.0 > > > > > > > >> > > PigVersion: 0.14.0 > > > > > >> TezVersion: 0.5.2 > > > > > > > >> > > UserId: dagriq > > > > > >> FileName: > > dev2/email4.pig > > > > > >> > > StartedAt: 2015-03-16 23:38:55 > > > > > > > >> > > FinishedAt: 2015-03-16 23:39:03 > > > > > >> Features: FILTER > > > > > > > >> > >> > > Failed! > >> > >> DAG > > PigLatin:dummy data-0_scope-0: > >> > > ApplicationId: null > > > > > >> TotalLaunchedTasks: -1 > > > > > > > >> > > FileBytesRead: -1 > > > > > >> FileBytesWritten: -1 > > > > > > > >> > > HdfsBytesRead: 0 > > > > > > > >> HdfsBytesWritten: > > 0 > > > > > >> > > > >> Input(s): > >> > >> Output(s): > >> > >> 2015-03-16 23:39:03,781 [main] ERROR > > org.apache.pig.tools.grunt.GruntParser - ERROR 2244: Job > > failed, hadoop does not return any error message > >> Details at logfile: > > /usr/local/pig/logs/pig-err.log > >> > > 2015-03-16 23:39:03,860 [main] INFO org.apache.pig.Main - > > Pig script completed in 19 seconds and 936 milliseconds > > (19936 ms) > >> 2015-03-16 23:39:03,860 > > [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher > > - Shutting down thread pool > >> > >> So I copied the tez-site.xml to the hadoop > > files and I re-executed the above mentioned command: pig -x > > tez myscript.pig. I received another error , I am copying it > > below: > >> > >> 2015-03-16 > > 23:40:28,679 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler > > - File concatenation threshold: 100 optimistic? false > >> 2015-03-16 23:40:28,805 [main] INFO > > org.apache.hadoop.conf.Configuration.deprecation - > > mapreduce.inputformat.class is deprecated. Instead, use > > mapreduce.job.inputformat.class > >> > > 2015-03-16 23:40:29,050 [main] INFO > > org.apache.tez.mapreduce.hadoop.MRInputHelpers - Generating > > mapreduce api input splits > >> 2015-03-16 > > 23:40:29,276 [main] INFO > > org.apache.hadoop.mapreduce.lib.input.FileInputFormat - > > Total input paths to process : 4 > >> > > 2015-03-16 23:40:29,280 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil > > - Total input paths to process : 4 > >> > > 2015-03-16 23:40:29,449 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil > > - Total input paths (combined) to process : 1 > >> 2015-03-16 23:40:29,602 [main] INFO > > org.apache.tez.mapreduce.hadoop.MRInputHelpers - NumSplits: > > 1, SerializedSize: 614 > >> 2015-03-16 > > 23:40:31,773 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler > > - Local resource: joda-time-2.1.jar > >> > > 2015-03-16 23:40:31,773 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler > > - Local resource: guava-11.0.2.jar > >> > > 2015-03-16 23:40:31,774 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler > > - Local resource: pig-0.14.0-core-h2.jar > >> 2015-03-16 23:40:31,774 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler > > - Local resource: antlr-runtime-3.4.jar > >> > > 2015-03-16 23:40:31,775 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler > > - Local resource: automaton-1.11-8.jar > >> > > 2015-03-16 23:40:31,884 [main] INFO > > org.apache.hadoop.conf.Configuration.deprecation - > > fs.default.name is deprecated. Instead, use fs.defaultFS > >> 2015-03-16 23:40:32,359 [main] INFO > > org.apache.hadoop.conf.Configuration.deprecation - > > mapred.output.compress is deprecated. Instead, use > > mapreduce.output.fileoutputformat.compress > >> 2015-03-16 23:40:32,365 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper > > - Setting tez.runtime.shuffle.merge.percent to 0.66 from MR > > setting mapreduce.reduce.shuffle.merge.percent > >> 2015-03-16 23:40:32,366 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper > > - Setting tez.runtime.shuffle.fetch.buffer.percent to 0.70 > > from MR setting > > mapreduce.reduce.shuffle.input.buffer.percent > >> 2015-03-16 23:40:32,367 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper > > - Setting tez.runtime.io.sort.mb to 100 from MR setting > > mapreduce.task.io.sort.mb > >> 2015-03-16 > > 23:40:32,367 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper > > - Setting tez.runtime.shuffle.memory.limit.percent to 0.25 > > from MR setting > > mapreduce.reduce.shuffle.memory.limit.percent > >> 2015-03-16 23:40:32,368 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper > > - Setting tez.runtime.io.sort.factor to 10 from MR setting > > mapreduce.task.io.sort.factor > >> > > 2015-03-16 23:40:32,368 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper > > - Setting tez.runtime.shuffle.connect.timeout to 180000 from > > MR setting mapreduce.reduce.shuffle.connect.timeout > >> 2015-03-16 23:40:32,369 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper > > - Setting tez.runtime.internal.sorter.class to > > org.apache.hadoop.util.QuickSort from MR setting > > map.sort.class > >> 2015-03-16 23:40:32,370 > > [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper > > - Setting tez.runtime.merge.progress.records to 10000 from > > MR setting mapreduce.task.merge.progress.records > >> 2015-03-16 23:40:32,370 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper > > - Setting tez.runtime.compress to false from MR setting > > mapreduce.map.output.compress > >> > > 2015-03-16 23:40:32,371 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper > > - Setting tez.runtime.sort.spill.percent to 0.80 from MR > > setting mapreduce.map.sort.spill.percent > >> 2015-03-16 23:40:32,372 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper > > - Setting tez.runtime.shuffle.ssl.enable to false from MR > > setting mapreduce.shuffle.ssl.enabled > >> > > 2015-03-16 23:40:32,373 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper > > - Setting tez.runtime.ifile.readahead to true from MR > > setting mapreduce.ifile.readahead > >> > > 2015-03-16 23:40:32,373 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper > > - Setting tez.runtime.shuffle.parallel.copies to 5 from MR > > setting mapreduce.reduce.shuffle.parallelcopies > >> 2015-03-16 23:40:32,374 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper > > - Setting tez.runtime.ifile.readahead.bytes to 4194304 from > > MR setting mapreduce.ifile.readahead.bytes > >> 2015-03-16 23:40:32,374 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper > > - Setting tez.runtime.task.input.post-merge.buffer.percent > > to 0.0 from MR setting > > mapreduce.reduce.input.buffer.percent > >> > > 2015-03-16 23:40:32,375 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper > > - Setting tez.runtime.shuffle.read.timeout to 180000 from MR > > setting mapreduce.reduce.shuffle.read.timeout > >> 2015-03-16 23:40:32,375 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.util.MRToTezHelper > > - Setting tez.runtime.compress.codec to > > org.apache.hadoop.io.compress.DefaultCodec from MR setting > > mapreduce.map.output.compress.codec > >> > > 2015-03-16 23:40:32,707 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler > > - For vertex - scope-34: parallelism=1, memory=1024, java > > opts=-Djava.net.preferIPv4Stack=true > > -Dhadoop.metrics.log.level=WARN -Xmx200m > > -Dlog4j.configuration=tez-container-log4j.properties > > -Dyarn.app.container.log.dir=<LOG_DIR> > > -Dtez.root.logger=INFO,CLA > >> 2015-03-16 > > 23:40:33,187 [PigTezLauncher-0] INFO > > org.apache.pig.tools.pigstats.tez.TezScriptState - Pig > > script settings are added to the job > >> > > 2015-03-16 23:40:33,899 [PigTezLauncher-0] INFO > > org.apache.tez.client.TezClient - Tez Client Version: [ > > component=tez-api, version=0.5.2, > > revision=5d728786b681e549a6f33375991dcce97cbf79f3, > > SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, > > buildTIme=20141104-1809 ] > >> > > 2015-03-16 23:40:34,098 [PigTezLauncher-0] INFO > > org.apache.hadoop.yarn.client.RMProxy - Connecting to > > ResourceManager at /0.0.0.0:8032 > >> > > 2015-03-16 23:40:34,923 [PigTezLauncher-0] INFO > > org.apache.tez.client.TezClient - Session mode. Starting > > session. > >> 2015-03-16 23:40:34,923 > > [PigTezLauncher-0] INFO org.apache.tez.client.TezClient - > > Session mode. Starting session. > >> > > 2015-03-16 23:40:34,924 [PigTezLauncher-0] INFO > > org.apache.tez.client.TezClientUtils - Using tez.lib.uris > > value from configuration: > > hdfs://localhost:9000/apps/tez-0.6.0-SNAPSHOT/tez-0.6.0.tar.gz > >> 2015-03-16 23:40:35,129 [PigTezLauncher-0] > > INFO org.apache.tez.client.TezClient - Tez system stage > > directory > > > hdfs://localhost:9000/tmp/temp-1303134320/.tez/application_1426563447689_0001 > > doesn't exist and is created > >> > > 2015-03-16 23:40:36,048 [PigTezLauncher-0] INFO > > org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - > > Submitted application application_1426563447689_0001 > >> 2015-03-16 23:40:36,055 [PigTezLauncher-0] > > INFO org.apache.tez.client.TezClient - The url to track > > the Tez Session: > http://optimf:8088/proxy/application_1426563447689_0001/ > >> > >> 2015-03-16 > > 23:40:59,709 [PigTezLauncher-0] ERROR > > org.apache.pig.backend.hadoop.executionengine.tez.TezJob - > > Cannot submit DAG > >> > > org.apache.tez.dag.api.SessionNotRunning: TezSession has > > already shutdown > >> at > > org.apache.tez.client.TezClient.waitTillReady(TezClient.java:603) > >> at > > > org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.createSession(TezSessionManager.java:100) > >> at > > > org.apache.pig.backend.hadoop.executionengine.tez.TezSessionManager.getClient(TezSessionManager.java:195) > >> at > > > org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:159) > >> at > > > org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:167) > >> at > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > >> at > > java.util.concurrent.FutureTask.run(FutureTask.java:262) > >> at > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > >> at > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > >> at > > java.lang.Thread.run(Thread.java:745) > >> > > 2015-03-16 23:41:00,014 [main] INFO > > org.apache.pig.tools.pigstats.tez.TezPigScriptStats - Script > > Statistics: > >> > >> > > HadoopVersion: 2.6.0 > > > > > > > >> > > PigVersion: 0.14.0 > > > > > >> TezVersion: 0.5.2 > > > > > > > >> > > UserId: dagriq > > > > > >> FileName: > > dev2/email4.pig > > > > > >> > > StartedAt: 2015-03-16 23:40:28 > > > > > > > >> > > FinishedAt: 2015-03-16 23:41:00 > > > > > >> Features: FILTER > > > > > > > >> > >> > > Failed! > >> > >> > >> DAG PigLatin:dummy data-0_scope-0: > >> ApplicationId: null > > > > > > > >> > > TotalLaunchedTasks: -1 > > > > > >> FileBytesRead: -1 > > > > > > > >> > > FileBytesWritten: -1 > > > > > >> HdfsBytesRead: 0 > > > > > > > >> > > HdfsBytesWritten: 0 > > > > > >> > >> Input(s): > >> > >> Output(s): > >> > >> 2015-03-16 > > 23:41:00,036 [main] ERROR > > org.apache.pig.tools.grunt.GruntParser - ERROR 2244: Job > > failed, hadoop does not return any error message > >> Details at logfile: > > /usr/local/pig/logs/pig-err.log > >> > > 2015-03-16 23:41:00,146 [main] INFO org.apache.pig.Main - > > Pig script completed in 42 seconds and 897 milliseconds > > (42897 ms) > >> 2015-03-16 23:41:00,147 > > [main] INFO > > org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher > > - Shutting down thread pool > >> > >> > >> What is wrong in my > > installation ? > >> Thank you for your > > help! > >> > > <2015-03-17-184721_1280x1024_scrot.png> > >
