[
https://issues.apache.org/jira/browse/OOZIE-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15800004#comment-15800004
]
Dongying Jiao commented on OOZIE-2767:
--------------------------------------
Hi:
Thanks for your information.
Yes, sparkr.zip should be put into workflow/lib dir.
And one property should be defined in workflow:
<configuration>
<property>
<name>oozie.launcher.mapred.child.env</name>
<value>SPARK_HOME=XXX</value>
</property>
</configuration>
Because SPARK_HOME is set to current dir only if this property is not defined.
After define this property, spark can find R lib.
> Oozie 4.3.0 with spark 2.0 run spark R job on yarn cluster mode failed
> -----------------------------------------------------------------------
>
> Key: OOZIE-2767
> URL: https://issues.apache.org/jira/browse/OOZIE-2767
> Project: Oozie
> Issue Type: Bug
> Components: action
> Affects Versions: 4.3.0
> Reporter: Dongying Jiao
> Attachments: spark1.sh, spark2.sh
>
>
> Erro seen from yarn log:
> Warning: Master yarn-cluster is deprecated since 2.0. Please use master
> "yarn" with specified deploy mode instead.
> Error: sparkr.zip does not exist for R application in YARN mode.
> Run with --help for usage help or --verbose for debug output
> Intercepting System.exit(1)
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.SparkMain], exit code [1]
> End of LogType:stderr
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
