[
https://issues.apache.org/jira/browse/SPARK-6797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14569432#comment-14569432
]
Shivaram Venkataraman commented on SPARK-6797:
----------------------------------------------
Yeah I don't think we can assume that SparkR is already installed on the AM
nodes. For example I don't think we assume that Spark assembly jar is present
in the AM nodes and right now we ship the assembly jar using --files if I am
not wrong. So what I had in mind was to zip the SparkR installed directory
(lib/SparkR) and then install it on the AM nodes. (I think we can assume that
SparkR is available on the node that spark-submit is running)
> Add support for YARN cluster mode
> ---------------------------------
>
> Key: SPARK-6797
> URL: https://issues.apache.org/jira/browse/SPARK-6797
> Project: Spark
> Issue Type: Improvement
> Components: SparkR
> Reporter: Shivaram Venkataraman
> Assignee: Sun Rui
> Priority: Critical
>
> SparkR currently does not work in YARN cluster mode as the R package is not
> shipped along with the assembly jar to the YARN AM. We could try to use the
> support for archives in YARN to send out the R package as a zip file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
