Hi As Zongheng mentioned we have been working on a R frontend that is similar in spirit to PySpark i.e it allows you to create and manipulate RDDs from R.
Our plan is to have an alpha version that the community can try out in the next few weeks -- We will ping the user list once its ready. Thanks Shivaram On Thu, Jan 2, 2014 at 6:14 PM, Zongheng Yang <[email protected]> wrote: > Hi Shay, > > Good to know there are interests in the R interface! I am not sure > about the specific release timeframe. Here's what should probably be > available in the initial release: > > - common RDD transformations (map() / lapply(), flatMap(), > lapplyPartition(), common suffle functions, ...) and actions (count() > / length(), collect(), ...) > - support for shipping R closures; support for pairwise RDDs > > The features are currently by no means complete yet, but the hope is > that the community could also contribute to it. So far we are able to > port some Spark examples and have them running, such as pi estimation, > logistic regression, and wordcount. > > Zongheng > > On Fri, Jan 3, 2014 at 1:36 AM, Shay Seng <[email protected]> wrote: >> I've been using JRI to communicate with R from Spark, with some utils to >> convert from Scala data types into R datatypes/dataframes etc. >> http://www.rforge.net/JRI/ >> I've been using mapPartitions to push R closures thru JRI and collecting >> back the results in Spark. This works reasonably well, though no where as >> nicely as straight Spark -- as expected. >> >> I've also been using JavaGD to allow me to use ggplot to visualize data from >> Spark -> R, that IMO, is much nicer than anything Java/scala can provide. >> >> >> It's interesting to hear of the R interface work at AMPLab, anyone there >> care to elaborate what will be available and the limitations and possible >> the timeframe? >> >> >> tks >> shay >> >> >> >> On Wed, Jan 1, 2014 at 8:55 PM, guxiaobo1982 <[email protected]> wrote: >>> >>> I read the good news from here: >>> >>> http://blog.revolutionanalytics.com/2013/12/apache-spark.html >>> >>> >>> >>> >> Currently, Spark supports programming interfaces for Scala, Java and >>> >> Python. >> For R users, there is good news: an R interface is in the >>> >> works >>> >> and under >>development by the team at AMPLab; our sources tell us this >>> >> is >>> >> expected to be >>released in the first half of 2014. >>> >>> >>> Regards, >>> >>> Xiaobo Gu >> >>
