Re: Unable to configure custom mapreduce.job.queuename in Oozie shell action

[Peter Cseh]

Hi,

Can you submit jobs into the specified queues outside Oozie?
Is there any reason Pig Action

can't
be used instead of Shell Action to submit Pig jobs?

Thanks,
Peter



On Sat, Sep 10, 2016 at 7:37 PM, Bijoy Deb 
wrote:

> Hi,
> I have configured 2 new queues 'launcher' and 'mapreduce' apart from
> 'default' in Capacity scheduler in yarn-site.
> Now, I am trying to submit a Pig job via *Oozie Shell action* into those
> queues queues such that the oozie launcher job goes into 'launcher' and pig
> job into 'mapreduce' queue.On submitting the oozie workflow I can see that
> my launcher vjob successfully runs in 'launcher' queue,but the
> Pig/mapreduce job is still running in default queue.
>
> Below is the snippet that I added to my workflow.xml for above:
>
> **
>
> **
>
> *oozie.launcher.mapreduce.job.queuename*
>
> *launcher*
>
> **
>
> **
>
> *mapreduce.job.queuename*
>
> *mapreduce*
> **
> **
>
> I also tried using 'mapred.job.queue.name' instead of
> 'mapreduce.job.queuename' above and also by passing it as
> -Dmapreduce.job.queuename=mapreduce while submitting the oozie job via
> Oozie cli.But in every case the Pig/mapreduce job goes into 'default'
> queue.
>
> Am I doing anything wrong here or is it that Oozie Shell action doesn't
> support 'mapreduce.job.quenename specification?
>
> I am using MR2, Hadoop version 2.7.1, oozie version 4.2.0.
>
> Any help would be really appreciated.
>
> Thanks
>



-- 
Peter Cseh
Software Engineer

Re: 答复: Launcher exception: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/conf/HiveConf

[Peter Cseh]

Hi,
I've just created a build from Oozie's master and checked and it seems
hive-exec jar is needed and included in hive2 action's sharelib:


> GezapetiMBP:bin gezapeti$ hdfs dfs -ls share/lib/lib_20160911145035/hive2
> Found 48 items
> ...
> -rw-r--r--   3 gezapeti supergroup 115618 2016-09-11 14:50
> share/lib/lib_20160911145035/hive2/hive-beeline-0.13.1.jar
> -rw-r--r--   3 gezapeti supergroup   15141449 2016-09-11 14:50
> share/lib/lib_20160911145035/hive2/hive-exec-0.13.1.jar

...

I don't know what's the situation for other Hive versions.

I've compiled Oozie with:
mvn clean package assembly:single -DskipTests -Puber -Dhadoop.version=2.4.0

Created sharelib using:

./oozie-setup.sh sharelib create -fs hdfs://localhost:9000 -locallib
${OOZIE_SRC_DIR}/sharelib/target/oozie-sharelib-4.3.0-SNAPSHOT/share/

If -locallib is not specified, the oozie-sharelib-version.tar.gz from
Oozie's home dir (defined by oozie.home.dir property) will be used.

Make sure that the sharelib version matches your Oozie and Hive versions.

Hope it helps,

gp


​