Please unsubscribe me from your mailing list. On Wed, May 1, 2019 at 12:32 PM Asim Zafir <asim.za...@gmail.com> wrote:
> ** appologies for sending multiple emails as the previous once got > truncated some how ** > Hi, > I am struggling to find the root cause of pig jobs not running in Cluster > Mode after we upgraded the cluster to use YARN (MRV2) from MRV1. When the > job is submitted, I see the following message as the most obvious > difference from the old cluster where the job was running fine on Cluster > mode. : > > On the new cluster (where Yarn is running) I see the following: It appears > the job is not getting submitted to the RM. I have ensure the PIG_HOME and > HADOOP_MAPRED_HOME is set right but despite of many tries this is what I am > getting. Since the job admission isn't successful on YARN its failing back > to LocalJobRunner Mode. > > Pig jobs were running fine with MRv1 but ever since the cluster was > upgraded to use YARN, this problem is being noticed. I have ensured the > classPATHs and environment variable are set right on this environment. We > are using Cloudera Distribution (CDH 5.10) and pig version 0.12 (Apache Pig > version 0.12.0-cdh5.10.2) > > Please assist with a solution - also do you think is there > any incompatibility with PIG running with YARN in the recent or previous > release of PIG or YARN? > > Thanks > Asim Zafir. > > 2019-05-01 17:37:23,485 [main] DEBUG org.apache.pig.impl.util.JarManager - > Adding entry pigContext to job jar > > 2019-05-01 17:37:23,721 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler > - jar file Job2836672534872456251.jar created > > 2019-05-01 17:37:23,721 [main] INFO > org.apache.hadoop.conf.Configuration.deprecation - mapred.jar is > deprecated. Instead, use mapreduce.job.jar > > 2019-05-01 17:37:23,742 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler > - Setting up single store job > > 2019-05-01 17:37:23,745 [main] DEBUG > org.apache.pig.data.SchemaTupleFrontend - Nothing registered to generate. > > *2019-05-01 17:37:23,773 [main] INFO > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 1 map-reduce job(s) waiting for submission.* > > 2019-05-01 17:37:23,774 [main] INFO > org.apache.hadoop.conf.Configuration.deprecation - > mapred.job.tracker.http.address is deprecated. Instead, use > mapreduce.jobtracker.http.address > > 2019-05-01 17:37:23,774 [main] INFO > org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is > deprecated. Instead, use mapreduce.jobtracker.address > > 2019-05-01 17:37:23,776 [JobControl] DEBUG > org.apache.pig.backend.hadoop23.PigJobControl - Checking state of job job > name: PigLatin:testrun.pig > > job id: job_pigexec_0 > > job state: WAITING > > job mapred id: null > > job message: just initialized > > job has no depending job: >
