Hi all, I'm trying to group X rows in a single one without shuffling the date.
I was thinking doing something like that : val myDF = Seq(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11).toDF("myColumn") myDF.withColumn("myColumn", expr("sliding(myColumn, 3)")) expected result: myColumn [1,2,3] [4,5,6] [7,8,9] [10, 11] Any insight on how to implement this? I saw in MlLib a SlidingRDD but I wanted to stay at Dataframe abstraction https://spark.apache.org/docs/1.3.1/api/java/org/apache/spark/mllib/rdd/SlidingRDD.html Thanks