Re: SparkR - calling as.vector() with rdd dataframe causes error

2015-09-22 Thread Luciano Resende
localDF is a pure R data frame and as.vector will work with no problems, as for calling it in the SparkR objects, try calling collect before you call as.vector (or in your case, the algorithms), that should solve your problem. On Mon, Sep 21, 2015 at 8:48 AM, Ellen Kraffmiller <

Re: SparkR - calling as.vector() with rdd dataframe causes error

2015-09-21 Thread Ellen Kraffmiller
Thank you for the link! I was using http://apache-spark-user-list.1001560.n3.nabble.com/, and I didn't see replies there. Regarding your code example, I'm doing the same thing and successfully creating the rdd, but the problem is that when I call a clustering algorithm like amap::hcluster(), I

Re: SparkR - calling as.vector() with rdd dataframe causes error

2015-09-18 Thread Ellen Kraffmiller
Thanks for your response. Is there a reason why this thread isn't appearing on the mailing list? So far, I only see my post, with no answers, although I have received 2 answers via email. It would be nice if other people could see these answers as well. On Thu, Sep 17, 2015 at 2:22 AM, Sun,

Re: SparkR - calling as.vector() with rdd dataframe causes error

2015-09-18 Thread Luciano Resende
I see the thread with all the responses on the bottom at mail-archive : https://www.mail-archive.com/user%40spark.apache.org/msg36882.html On Fri, Sep 18, 2015 at 7:58 AM, Ellen Kraffmiller < ellen.kraffmil...@gmail.com> wrote: > Thanks for your response. Is there a reason why this thread

RE: SparkR - calling as.vector() with rdd dataframe causes error

2015-09-17 Thread Sun, Rui
The existing algorithms operating on R data.frame can't simply operate on SparkR DataFrame. They have to be re-implemented to be based on SparkR DataFrame API. -Original Message- From: ekraffmiller [mailto:ellen.kraffmil...@gmail.com] Sent: Thursday, September 17, 2015 3:30 AM To:

Re: SparkR - calling as.vector() with rdd dataframe causes error

2015-09-17 Thread Luciano Resende
You can find some more info about SparkR at https://spark.apache.org/docs/latest/sparkr.html Looking at your sample app, with the provided content, you should be able to run it on SparkR with something like: #load SparkR with support for csv sparkR --packages com.databricks:spark-csv_2.10:1.0.3

Re: SparkR - calling as.vector() with rdd dataframe causes error

2015-09-16 Thread ekraffmiller
Also, just for completeness, matrix.csv contains: 1,2,3 4,5,6 7,8,9 -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/SparkR-calling-as-vector-with-rdd-dataframe-causes-error-tp24717p24719.html Sent from the Apache Spark User List mailing list archive at