Hello !
I'm trying to use tez in a pig action of an oozie workflow.
The workflow itself is pretty simple :
###
<workflow-app xmlns="uri:oozie:workflow:0.5" name="TEST_PIG_ACTION">
<start to="init-pig" />
<action name="init-pig">
<pig>
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<prepare>
<delete path="/tmp/test/output.txt"/>
</prepare>
<configuration>
<property>
<name>mapreduce.job.queuename</name>
<value>${queueName}</value>
</property>
</configuration>
<script>pig.pig</script>
<file>pig.pig#pig.pig</file>
</pig>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Script failed, error
message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>
###
The properties :
###
#
------------------------------------------------------------------------------
# Environment
#
------------------------------------------------------------------------------
nameNode=hdfs://<NN FQDN>:8020
jobTracker=<RM FQDN>:8050
kerberosRealm=<REALM>
queueName=<QUEUE>
# Application
#
------------------------------------------------------------------------------
appRoot=${nameNode}/tmp/test
oozie.wf.application.path=${appRoot}/pig.xml
#
------------------------------------------------------------------------------
# Oozie
#
------------------------------------------------------------------------------
oozie.use.system.libpath=true
oozie.wf.rerun.failnodes=true
###
The pig script :
###
data = LOAD '/tmp/test/data.txt' USING PigStorage(',') AS (user, age,
salary);
filtered_data = FILTER data BY age > 11;
ordered_data = ORDER filtered_data BY salary;
final_data = FOREACH ordered_data GENERATE (user, age, salary);
STORE final_data INTO '/tmp/test/output.txt' USING PigStorage();
###
This workflow, launched in MR works.
But as soon as I try to execute it with Tez, it fails.
I tried the following configuration to make it work with Tez, but it fails :
###
<property>
<name>exectype</name>
<value>tez</value>
</property>
<property>
<name>tez.queue.name</name>
<value>${queueName}</value>
</property>
###
When I added these properties, it failed because it does take into account
the property tez.queue.name.
So my first question would be : how can I specify that I want to use the
execution engine tez instead of MR in a pig action in an oozie workflow ?
And then how can I specify the queue I want to use ?
BR.
Morgrim.
