I say you need to remap so you have a key for each tuple that you can sort on. Then call rdd.sortByKey(true) like this mystream.transform(rdd => rdd.sortByKey(true)) For this fn to be available you need to import org.apache.spark.rdd.OrderedRDDFunctions
-----Original Message----- From: yh18190 [mailto:yh18...@gmail.com] Sent: March-28-14 5:02 PM To: u...@spark.incubator.apache.org Subject: RE: Splitting RDD and Grouping together to perform computation Hi, Here is my code for given scenario.Could you please let me know where to sort?I mean on what basis we have to sort??so that they maintain order in partition as thatof original sequence.. val res2=reduced_hccg.map(_._2)// which gives RDD of numbers res2.foreach(println) val result= res2.mapPartitions(p=>{ val l=p.toList val approx=new ListBuffer[(Int)] val detail=new ListBuffer[Double] for(i<-0 until l.length-1 by 2) { println(l(i),l(i+1)) approx+=(l(i),l(i+1)) } approx.toList.iterator detail.toList.iterator }) result.foreach(println) -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Splitting-RDD-and-Grouping-together-to-perform-computation-tp3153p3450.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
