Hi, I have a streaming application where am doing top 10 count in each window which seems slow. Is there efficient way to do this. val counts = keyAndValues.map(x => math.round(x._3.toDouble)).countByValueAndWindow(Seconds(4), Seconds(4)) val topCounts = counts.repartition(1).map(_.swap).transform(rdd => rdd.sortByKey(false)).map(_.swap).mapPartitions(rdd => rdd.take(10)) Regards, Laeeq
- Efficient Top count in each window Laeeq Ahmed
- Re: Efficient Top count in each window Tathagata Das