[jira] [Commented] (SPARK-8724) Need documentation on how to deploy or use SparkR in Spark 1.4.0+
[ https://issues.apache.org/jira/browse/SPARK-8724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741007#comment-14741007 ] Deborah Siegel commented on SPARK-8724: --- I would work on this documentation if thats fine. > Need documentation on how to deploy or use SparkR in Spark 1.4.0+ > - > > Key: SPARK-8724 > URL: https://issues.apache.org/jira/browse/SPARK-8724 > Project: Spark > Issue Type: Bug > Components: R >Affects Versions: 1.4.0 >Reporter: Felix Cheung >Priority: Minor > > As of now there doesn't seem to be any official documentation on how to > deploy SparkR with Spark 1.4.0+ > Also, cluster manager specific documentation (like > http://spark.apache.org/docs/latest/spark-standalone.html) does not call out > what mode is supported for SparkR and details on deployment steps. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8724) Need documentation on how to deploy or use SparkR in Spark 1.4.0+
[ https://issues.apache.org/jira/browse/SPARK-8724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681427#comment-14681427 ] Vincent Warmerdam commented on SPARK-8724: -- [~shivaram] [~felixcheung] does this still need to be open? or do we want to add parts of the rstudio blog post to documentation on the sparkr end? Need documentation on how to deploy or use SparkR in Spark 1.4.0+ - Key: SPARK-8724 URL: https://issues.apache.org/jira/browse/SPARK-8724 Project: Spark Issue Type: Bug Components: R Affects Versions: 1.4.0 Reporter: Felix Cheung Priority: Minor As of now there doesn't seem to be any official documentation on how to deploy SparkR with Spark 1.4.0+ Also, cluster manager specific documentation (like http://spark.apache.org/docs/latest/spark-standalone.html) does not call out what mode is supported for SparkR and details on deployment steps. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8724) Need documentation on how to deploy or use SparkR in Spark 1.4.0+
[ https://issues.apache.org/jira/browse/SPARK-8724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682219#comment-14682219 ] Shivaram Venkataraman commented on SPARK-8724: -- [~cantdutchthis] One thing we could do is add a section at the bottom of http://spark.apache.org/docs/latest/sparkr.html titled `Deploying SparkR` or `Where to go from here` and a short description of how to launch EC2 clusters with RStudio (in 1.5) and also link to the RStudio blog post. Need documentation on how to deploy or use SparkR in Spark 1.4.0+ - Key: SPARK-8724 URL: https://issues.apache.org/jira/browse/SPARK-8724 Project: Spark Issue Type: Bug Components: R Affects Versions: 1.4.0 Reporter: Felix Cheung Priority: Minor As of now there doesn't seem to be any official documentation on how to deploy SparkR with Spark 1.4.0+ Also, cluster manager specific documentation (like http://spark.apache.org/docs/latest/spark-standalone.html) does not call out what mode is supported for SparkR and details on deployment steps. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8724) Need documentation on how to deploy or use SparkR in Spark 1.4.0+
[ https://issues.apache.org/jira/browse/SPARK-8724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14626482#comment-14626482 ] Vincent Warmerdam commented on SPARK-8724: -- So a tutorial just went live for people who are on spark 1.4. http://blog.rstudio.org/2015/07/14/spark-1-4-for-rstudio/ I suppose if people link to this for now it'd be just fine. For spark 1.5 the provisioning script of ec2 will come with rstudio. Need documentation on how to deploy or use SparkR in Spark 1.4.0+ - Key: SPARK-8724 URL: https://issues.apache.org/jira/browse/SPARK-8724 Project: Spark Issue Type: Bug Components: R Affects Versions: 1.4.0 Reporter: Felix Cheung Priority: Minor As of now there doesn't seem to be any official documentation on how to deploy SparkR with Spark 1.4.0+ Also, cluster manager specific documentation (like http://spark.apache.org/docs/latest/spark-standalone.html) does not call out what mode is supported for SparkR and details on deployment steps. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8724) Need documentation on how to deploy or use SparkR in Spark 1.4.0+
[
https://issues.apache.org/jira/browse/SPARK-8724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615007#comment-14615007
]
Vincent Warmerdam commented on SPARK-8724:
--
```
spark_link - system('cat /root/spark-ec2/cluster-url', intern=TRUE)
.libPaths(c(.libPaths(), '/root/spark/R/lib'))
Sys.setenv(SPARK_HOME = '/root/spark')
Sys.setenv(PATH = paste(Sys.getenv(c('PATH')), '/root/spark/bin', sep=':'))
library(SparkR)
sc - sparkR.init(spark_link)
sqlContext - sparkRSQL.init(sc)
cat('Spark Context available as \sc\. \\n')
cat('Spark SQL Context available as \sqlContext\. \\n')
```
A tutorial on how to do this will soon be available on the Rstudio blog as
well.
Need documentation on how to deploy or use SparkR in Spark 1.4.0+
-
Key: SPARK-8724
URL: https://issues.apache.org/jira/browse/SPARK-8724
Project: Spark
Issue Type: Bug
Components: R
Affects Versions: 1.4.0
Reporter: Felix Cheung
Priority: Minor
As of now there doesn't seem to be any official documentation on how to
deploy SparkR with Spark 1.4.0+
Also, cluster manager specific documentation (like
http://spark.apache.org/docs/latest/spark-standalone.html) does not call out
what mode is supported for SparkR and details on deployment steps.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8724) Need documentation on how to deploy or use SparkR in Spark 1.4.0+
[ https://issues.apache.org/jira/browse/SPARK-8724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610589#comment-14610589 ] Shivaram Venkataraman commented on SPARK-8724: -- Sure. We could add that to the examples. I also think we could also add some more details on how to launch SparkR if you are not using the script bin/sparkR (i.e. from RStudio or from plain R). Need documentation on how to deploy or use SparkR in Spark 1.4.0+ - Key: SPARK-8724 URL: https://issues.apache.org/jira/browse/SPARK-8724 Project: Spark Issue Type: Bug Components: R Affects Versions: 1.4.0 Reporter: Felix Cheung Priority: Minor As of now there doesn't seem to be any official documentation on how to deploy SparkR with Spark 1.4.0+ Also, cluster manager specific documentation (like http://spark.apache.org/docs/latest/spark-standalone.html) does not call out what mode is supported for SparkR and details on deployment steps. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8724) Need documentation on how to deploy or use SparkR in Spark 1.4.0+
[ https://issues.apache.org/jira/browse/SPARK-8724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609673#comment-14609673 ] Felix Cheung commented on SPARK-8724: - For instance, http://spark.apache.org/docs/latest/sparkr.html has no mention of `library(SparkR)` without which `sc - sparkR.init()` would not work? Need documentation on how to deploy or use SparkR in Spark 1.4.0+ - Key: SPARK-8724 URL: https://issues.apache.org/jira/browse/SPARK-8724 Project: Spark Issue Type: Bug Components: R Affects Versions: 1.4.0 Reporter: Felix Cheung Priority: Minor As of now there doesn't seem to be any official documentation on how to deploy SparkR with Spark 1.4.0+ Also, cluster manager specific documentation (like http://spark.apache.org/docs/latest/spark-standalone.html) does not call out what mode is supported for SparkR and details on deployment steps. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8724) Need documentation on how to deploy or use SparkR in Spark 1.4.0+
[ https://issues.apache.org/jira/browse/SPARK-8724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609666#comment-14609666 ] Felix Cheung commented on SPARK-8724: - Doesn't SparkR need to be installed in R for `library(SparkR)` in this example to work? https://github.com/apache/spark/blob/master/examples/src/main/r/dataframe.R This JIRA is referring to that if someone is deploying Spark himself (for different cluster manager), how does he get a sample with SparkR to work? Need documentation on how to deploy or use SparkR in Spark 1.4.0+ - Key: SPARK-8724 URL: https://issues.apache.org/jira/browse/SPARK-8724 Project: Spark Issue Type: Bug Components: R Affects Versions: 1.4.0 Reporter: Felix Cheung Priority: Minor As of now there doesn't seem to be any official documentation on how to deploy SparkR with Spark 1.4.0+ Also, cluster manager specific documentation (like http://spark.apache.org/docs/latest/spark-standalone.html) does not call out what mode is supported for SparkR and details on deployment steps. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8724) Need documentation on how to deploy or use SparkR in Spark 1.4.0+
[ https://issues.apache.org/jira/browse/SPARK-8724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14608573#comment-14608573 ] Shivaram Venkataraman commented on SPARK-8724: -- I'm not sure what kind of documentation we need -- could you explain more ? Other than YARN cluster mode SparkR should work in all other modes by just running bin/sparkR (for shell) and bin/spark-submit (for batch jobs). Feel free to open a PR if you have a good idea of what would be useful. Need documentation on how to deploy or use SparkR in Spark 1.4.0+ - Key: SPARK-8724 URL: https://issues.apache.org/jira/browse/SPARK-8724 Project: Spark Issue Type: Bug Components: R Affects Versions: 1.4.0 Reporter: Felix Cheung Priority: Minor As of now there doesn't seem to be any official documentation on how to deploy SparkR with Spark 1.4.0+ Also, cluster manager specific documentation (like http://spark.apache.org/docs/latest/spark-standalone.html) does not call out what mode is supported for SparkR and details on deployment steps. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
