Hello Satish.
Thank you for your answer.
I added the argument, but I performed two other modifications.
Here they are :
1. First in the configuration tag of my xml file, I added two properties :
<property>
<name>tez.lib.uris</name>
<value>/hdp/apps/2.3.2.0-2950/tez/tez.tar.gz</value>
</property>
<property>
<name>mapreduce.job.queuename</name>
<value>${queueName}</value>
</property>
But it was not enough. I was still encountering error about classes not
found.
2. Second in the pig folder of the oozie sharelib
I added the tez libs in the pig folder of the oozie sharelibs, and then
update them.
I'm using HDP as distribution, so my tez libs are in /usr/hdp/<hdp.version>.
###
# From any nodes of the cluster
kinit -kt /etc/security/keytabs/hdfs.headless.keytab hdfs
hdfs dfs -put /usr/hdp/2.3.2.0-2950/tez/lib/*
/user/oozie/share/lib/lib_*/pig/
hdfs dfs -put /usr/hdp/2.3.2.0-2950/tez/*.jar
/user/oozie/share/lib/lib_*/pig/
hdfs dfs -chown -R oozie /user/oozie/share/lib/lib_<timestamp>/pig
hdfs dfs -chmod -R 755 /user/oozie/share/lib/lib_<timestamp>/pig
# On the host with oozie-server
kinit -kt /etc/security/keytabs/oozie.service.keytab oozie/`hostname
-f`@<REALM>
oozie admin -oozie http://<fqdn oozie-server> <http://%3CFQDN>:11000/oozie
-sharelibupdate
###
And then I was able to run my worklow successfully.
I'm really wondering why I have to use mapreduce.job.queuename for my tez
job instead of tez.queue.name.
And I wonder why the tez libs where not already in the pig folder of the
oozie sharelibs.
Anyway ty again for your help.
BR.
Morgrim.
On Thu, Apr 28, 2016 at 7:33 PM, satish saley <[email protected]>
wrote:
> You can specify execution engine using argument tag -
>
> <argument>-x</argument>
> <argument>tez</argument>
>
>
>
> On Thu, Apr 28, 2016 at 8:26 AM, Morgrim Murdargent <
> [email protected]
> > wrote:
>
> > Has anyone any idea about this please ?
> >
> > BR.
> >
> > Morgrim.
> >
> > On Tue, Apr 26, 2016 at 5:22 PM, Morgrim Murdargent <
> > [email protected]
> > > wrote:
> >
> > > Hello !
> > >
> > > I'm trying to use tez in a pig action of an oozie workflow.
> > >
> > > The workflow itself is pretty simple :
> > > ###
> > > <workflow-app xmlns="uri:oozie:workflow:0.5" name="TEST_PIG_ACTION">
> > > <start to="init-pig" />
> > > <action name="init-pig">
> > > <pig>
> > > <job-tracker>${jobTracker}</job-tracker>
> > > <name-node>${nameNode}</name-node>
> > > <prepare>
> > > <delete path="/tmp/test/output.txt"/>
> > > </prepare>
> > > <configuration>
> > > <property>
> > > <name>mapreduce.job.queuename</name>
> > > <value>${queueName}</value>
> > > </property>
> > > </configuration>
> > > <script>pig.pig</script>
> > > <file>pig.pig#pig.pig</file>
> > > </pig>
> > > <ok to="end"/>
> > > <error to="fail"/>
> > > </action>
> > >
> > > <kill name="fail">
> > > <message>Script failed, error
> > > message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> > > </kill>
> > >
> > > <end name="end"/>
> > > </workflow-app>
> > > ###
> > >
> > > The properties :
> > > ###
> > > #
> > >
> >
> ------------------------------------------------------------------------------
> > > # Environment
> > > #
> > >
> >
> ------------------------------------------------------------------------------
> > > nameNode=hdfs://<NN FQDN>:8020
> > > jobTracker=<RM FQDN>:8050
> > > kerberosRealm=<REALM>
> > > queueName=<QUEUE>
> > >
> > > # Application
> > > #
> > >
> >
> ------------------------------------------------------------------------------
> > > appRoot=${nameNode}/tmp/test
> > > oozie.wf.application.path=${appRoot}/pig.xml
> > >
> > > #
> > >
> >
> ------------------------------------------------------------------------------
> > > # Oozie
> > > #
> > >
> >
> ------------------------------------------------------------------------------
> > > oozie.use.system.libpath=true
> > > oozie.wf.rerun.failnodes=true
> > > ###
> > >
> > > The pig script :
> > > ###
> > > data = LOAD '/tmp/test/data.txt' USING PigStorage(',') AS (user, age,
> > > salary);
> > > filtered_data = FILTER data BY age > 11;
> > > ordered_data = ORDER filtered_data BY salary;
> > > final_data = FOREACH ordered_data GENERATE (user, age, salary);
> > > STORE final_data INTO '/tmp/test/output.txt' USING PigStorage();
> > > ###
> > >
> > >
> > > This workflow, launched in MR works.
> > > But as soon as I try to execute it with Tez, it fails.
> > > I tried the following configuration to make it work with Tez, but it
> > fails
> > > :
> > > ###
> > > <property>
> > > <name>exectype</name>
> > > <value>tez</value>
> > > </property>
> > > <property>
> > > <name>tez.queue.name</name>
> > > <value>${queueName}</value>
> > > </property>
> > > ###
> > >
> > > When I added these properties, it failed because it does take into
> > account
> > > the property tez.queue.name.
> > >
> > > So my first question would be : how can I specify that I want to use
> the
> > > execution engine tez instead of MR in a pig action in an oozie
> workflow ?
> > >
> > > And then how can I specify the queue I want to use ?
> > >
> > > BR.
> > >
> > > Morgrim.
> > >
> >
>
