[
https://issues.apache.org/jira/browse/OOZIE-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15798399#comment-15798399
]
Peter Cseh commented on OOZIE-2767:
-----------------------------------
It seems you'd like to run SparkR jobs from Oozie.
Oozie 4.3.0 can be compiled with Spark2, but the Python and R scripts need some
manual work.
The [error
message|https://github.com/sun-rui/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L379-L382]
arises when the sparkr.zip can't be found at the location
[SPARK_HOME/R/lib|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/api/r/RUtils.scala#L33-L38].
Probably it have to added to the sharelib under the required folder.
> Oozie 4.3.0 with spark 2.0 run spark R job on yarn cluster mode failed
> -----------------------------------------------------------------------
>
> Key: OOZIE-2767
> URL: https://issues.apache.org/jira/browse/OOZIE-2767
> Project: Oozie
> Issue Type: Bug
> Components: action
> Affects Versions: 4.3.0
> Reporter: Dongying Jiao
> Attachments: spark1.sh, spark2.sh
>
>
> Erro seen from yarn log:
> Warning: Master yarn-cluster is deprecated since 2.0. Please use master
> "yarn" with specified deploy mode instead.
> Error: sparkr.zip does not exist for R application in YARN mode.
> Run with --help for usage help or --verbose for debug output
> Intercepting System.exit(1)
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.SparkMain], exit code [1]
> End of LogType:stderr
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
