[
https://issues.apache.org/jira/browse/OOZIE-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Robertson updated OOZIE-2066:
---------------------------------
Attachment: test-wf.zip
In the example attached, a simple Java action uses a Guava method that requires
Guava 17.0+. Hadoop (CDH5.2) will bring in 11.0.2, and thus this demonstrates
that it fails as it picks up the ancient version. The workaround above can be
used to show how it can be circumvented.
To build, install and run the example, requires maven and a working Hadoop
(Yarn) environment:
{code}
mvn clean package
hadoop fs -put target/workflow-hello-world /user/tim/workflow-hello-world
oozie job -oozie http://c1n2.gbif.org:11000/oozie/ -config
example-jobs/hello-world.xml -run
{code}
The job will succeed but the logs for the task will show:
{code}
Calling a Guava 17+ method!
FAIL:
com.google.common.io.ByteStreams.newDataOutput(Ljava/io/ByteArrayOutputStream;)Lcom/google/common/io/ByteArrayDataOutput;
{code}
Adding the workaround above will succeed.
> oozie.launcher.mapreduce.task.classpath.user.precedence is not respected
> ------------------------------------------------------------------------
>
> Key: OOZIE-2066
> URL: https://issues.apache.org/jira/browse/OOZIE-2066
> Project: Oozie
> Issue Type: Bug
> Environment: yarn
> Reporter: Tim Robertson
> Priority: Critical
> Attachments: test-wf.zip
>
>
> When using MR2, the user classpath is not read from the job configuration.
> When submitting a job, the following configuration should result in the java
> action running with the user classpath before the Hadoop jars.
> {code:xml}
> <property>
> <name>oozie.launcher.mapreduce.task.classpath.user.precedence</name>
> <value>true</value>
> </property>
> {code}
> When used in a Java action:
> {code:xml}
> <action name="run-test">
> <java>
> <job-tracker>c1n2.gbif.org:8032</job-tracker>
> <name-node>hdfs://c1n1.gbif.org:8020</name-node>
> <main-class>test.CPTest</main-class>
> </java>
> <ok to="end" />
> <error to="kill" />
> </action>
> {code}
> However, it is not...
> There is a workaround, by setting this on the task directly in the workflow:
> {code:xml}
> <action name="run-test">
> <java>
> <job-tracker>c1n2.gbif.org:8032</job-tracker>
> <name-node>hdfs://c1n1.gbif.org:8020</name-node>
> <configuration>
> <property>
> <name>oozie.launcher.mapreduce.task.classpath.user.precedence</name>
> <value>true</value>
> </property>
> </configuration>
> <main-class>test.CPTest</main-class>
> </java>
> <ok to="end" />
> <error to="kill" />
> </action>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
