Re: In Spark-Scala, how to copy Array of Lists into new DataFrame?

2016-09-25 Thread Marco Mistroni
Hi in fact i have just found some written notes in my code see if this docs help you (it will work with any spark versions, not only 1.3.0) https://spark.apache.org/docs/1.3.0/sql-programming-guide.html#creating-dataframes hth On Sun, Sep 25, 2016 at 1:25 PM, Marco Mistroni

Re: In Spark-Scala, how to copy Array of Lists into new DataFrame?

2016-09-25 Thread Marco Mistroni
Hi i must admit , i had issues as well in finding a sample that does that, (hopefully Spark folks can add more examples or someone on the list can post a sample code?) hopefully you can reuse sample below So, you start from an rdd of doubles (myRdd) ## make a row val toRddOfRows =

In Spark-Scala, how to copy Array of Lists into new DataFrame?

2016-09-25 Thread Dan Bikle
Hello World, I am familiar with Python and I am learning Spark-Scala. I want to build a DataFrame which has structure desribed by this syntax: *// Prepare training data from a list of (label, features) tuples.val training = spark.createDataFrame(Seq( (1.1, Vectors.dense(1.1, 0.1)),