Hi, When I submit a job using Pig command,I am able to submit it to the custom queue using 'set mapreduce.job.queuename <name_of_queue>' command inside Pig script.It seems the problem is when I specify the queuename in <configuration> tag via Oozie shell action.
We cannot use the Pig script directly and use shell action instead as we have some custom operations such as logging,auditing,etc. that we want to perform via the shell script during the course of execution of the Pig job from within it. Thanks On Mon, Sep 12, 2016 at 1:44 AM, Peter Cseh <[email protected]> wrote: > Hi, > > Can you submit jobs into the specified queues outside Oozie? > Is there any reason Pig Action > <https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html# > a3.2.3_Pig_Action> > can't > be used instead of Shell Action to submit Pig jobs? > > Thanks, > Peter > > > > On Sat, Sep 10, 2016 at 7:37 PM, Bijoy Deb <[email protected]> > wrote: > > > Hi, > > I have configured 2 new queues 'launcher' and 'mapreduce' apart from > > 'default' in Capacity scheduler in yarn-site. > > Now, I am trying to submit a Pig job via *Oozie Shell action* into those > > queues queues such that the oozie launcher job goes into 'launcher' and > pig > > job into 'mapreduce' queue.On submitting the oozie workflow I can see > that > > my launcher vjob successfully runs in 'launcher' queue,but the > > Pig/mapreduce job is still running in default queue. > > > > Below is the snippet that I added to my workflow.xml for above: > > > > *<configuration>* > > > > *<property>* > > > > *<name>oozie.launcher.mapreduce.job.queuename</name>* > > > > *<value>launcher</value>* > > > > *</property>* > > > > *<property>* > > > > *<name>mapreduce.job.queuename</name>* > > > > *<value>mapreduce</value>* > > *</property>* > > *</configuration>* > > > > I also tried using 'mapred.job.queue.name' instead of > > 'mapreduce.job.queuename' above and also by passing it as > > -Dmapreduce.job.queuename=mapreduce while submitting the oozie job via > > Oozie cli.But in every case the Pig/mapreduce job goes into 'default' > > queue. > > > > Am I doing anything wrong here or is it that Oozie Shell action doesn't > > support 'mapreduce.job.quenename specification? > > > > I am using MR2, Hadoop version 2.7.1, oozie version 4.2.0. > > > > Any help would be really appreciated. > > > > Thanks > > > > > > -- > Peter Cseh > Software Engineer > <http://www.cloudera.com> >
