[
https://issues.apache.org/jira/browse/SPARK-13573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15175093#comment-15175093
]
Chip Senkbeil commented on SPARK-13573:
---------------------------------------
I'd gladly create a PR with the changes if needed. We haven't synced with Spark
1.6.0+ yet, so it'd just take me a little bit to get up to speed. Other than
the one new method to enable connecting without creating a Spark Context, it's
just exporting functions and switching the RBackend class to be public.
> Open SparkR APIs (R package) to allow better 3rd party usage
> ------------------------------------------------------------
>
> Key: SPARK-13573
> URL: https://issues.apache.org/jira/browse/SPARK-13573
> Project: Spark
> Issue Type: Improvement
> Components: SparkR
> Reporter: Chip Senkbeil
>
> Currently, SparkR's R package does not expose enough of its APIs to be used
> flexibly. That I am aware of, SparkR still requires you to create a new
> SparkContext by invoking the sparkR.init method (so you cannot connect to a
> running one) and there is no way to invoke custom Java methods using the
> exposed SparkR API (unlike PySpark).
> We currently maintain a fork of SparkR that is used to power the R
> implementation of Apache Toree, which is a gateway to use Apache Spark. This
> fork provides a connect method (to use an existing Spark Context), exposes
> needed methods like invokeJava (to be able to communicate with our JVM to
> retrieve code to run, etc), and uses reflection to access
> org.apache.spark.api.r.RBackend.
> Here is the documentation I recorded regarding changes we need to enable
> SparkR as an option for Apache Toree:
> https://github.com/apache/incubator-toree/tree/master/sparkr-interpreter/src/main/resources
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
