[
https://issues.apache.org/jira/browse/OOZIE-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16086594#comment-16086594
]
Vivek Kapoor commented on OOZIE-2767:
-------------------------------------
>From the application's log:
*stdout:*
{quote}Loading required package: methods
Attaching package: ‘SparkR’
The following objects are masked from ‘package:stats’:
cov, filter, lag, na.omit, predict, sd, var, window
The following objects are masked from ‘package:base’:
as.data.frame, colnames, colnames<-, drop, endsWith, intersect,
rank, rbind, sample, startsWith, subset, summary, transform, union
Warning message:
'sparkR.init' is deprecated.
Use 'sparkR.session' instead.
See help("Deprecated")
Error in handleErrors(returnStatus, conn) :
Calls: sparkRSQL.init ... sparkR.session -> callJStatic -> invokeJava ->
handleErrors
In addition: Warning message:
'sparkRSQL.init' is deprecated.
Use 'sparkR.session' instead.
See help("Deprecated")
Execution halted{quote}
-------------------
*stderr*
{quote}17/07/13 23:14:17 INFO YarnClusterSchedulerBackend: SchedulerBackend is
ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.8
17/07/13 23:14:17 INFO YarnClusterScheduler: YarnClusterScheduler.postStartHook
done
17/07/13 23:14:17 ERROR RBackendHandler: getOrCreateSparkSession on
org.apache.spark.sql.api.r.SQLUtils failed
17/07/13 23:14:18 ERROR ApplicationMaster: User application exited with status 1
17/07/13 23:14:18 INFO ApplicationMaster: Final app status: FAILED, exitCode:
1, (reason: User application exited with status 1)
17/07/13 23:14:18 INFO SparkContext: Invoking stop() from shutdown hook{quote}
> Oozie 4.3.0 with spark 2.0 run spark R job on yarn cluster mode failed
> -----------------------------------------------------------------------
>
> Key: OOZIE-2767
> URL: https://issues.apache.org/jira/browse/OOZIE-2767
> Project: Oozie
> Issue Type: Bug
> Components: action
> Affects Versions: 4.3.0
> Reporter: Dongying Jiao
> Attachments: spark1.sh, spark2.sh
>
>
> Erro seen from yarn log:
> Warning: Master yarn-cluster is deprecated since 2.0. Please use master
> "yarn" with specified deploy mode instead.
> Error: sparkr.zip does not exist for R application in YARN mode.
> Run with --help for usage help or --verbose for debug output
> Intercepting System.exit(1)
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.SparkMain], exit code [1]
> End of LogType:stderr
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
