GitHub user sun-rui opened a pull request:
https://github.com/apache/spark/pull/6743
[SPARK-6797][SPARKR] Add support for YARN cluster mode.
This PR enables SparkR to dynamically ship the SparkR binary package to the
AM node in YARN cluster mode, thus it is no longer required that the SparkR
package be installed on each worker node.
This PR uses the JDK jar tool to package the SparkR package, because jar is
thought to be available on both Linux/Windows platforms where JDK has been
installed.
This PR does not address the R worker involved in RDD API. Will address it
in a separate JIRA issue.
This PR does not address SBT build. SparkR installation and packaging by
SBT will be addressed in a separate JIRA issue.
R/install-dev.bat is not tested. @shivaram , Could you help to test it?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sun-rui/spark SPARK-6797
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/6743.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6743
----
commit 528f30e619619d9537ee9a6839d1a891f50c734b
Author: Sun Rui <[email protected]>
Date: 2015-06-10T11:26:02Z
[SPARK-6797][SPARKR] Add support for YARN cluster mode.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
