Thanks. It works for me. From: mailforledkk [mailto:mailforle...@126.com] Sent: Wednesday, April 09, 2014 9:16 AM To: user Cc: mailforledkk Subject: Re: To Ten RDD
i see the top method in RDD class , you can use this method to get top N , but i found some error when i use this method .... it's seem as when mapPartitions in top , the task result may be an Nil , then reduce will result an class cast exception as blow : java.lang.ClassCastException: scala.collection.immutable.Nil$ cannot be cast to org.apache.spark.util.BoundedPriorityQueue at org.apache.spark.rdd.RDD$$anonfun$top$2.apply(RDD.scala:873) at org.apache.spark.rdd.RDD$$anonfun$6.apply(RDD.scala:671) at org.apache.spark.rdd.RDD$$anonfun$6.apply(RDD.scala:668) at org.apache.spark.scheduler.JobWaiter.taskSucceeded(JobWaiter.scala:56) at org.apache.spark.scheduler.DAGScheduler.handleTaskCompletion(DAGScheduler.scala:859) at org.apache.spark.scheduler.DAGScheduler.processEvent(DAGScheduler.scala:616) at org.apache.spark.scheduler.DAGScheduler$$anonfun$start$1$$anon$2$$anonfun$receive$1.applyOrElse(DAGScheduler.scala:207) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498) at akka.actor.ActorCell.invoke(ActorCell.scala:456) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237) at akka.dispatch.Mailbox.run(Mailbox.scala:219) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) my spark version is : 0.9.0 , i don't konw is this a bug of spark .. there are another method to get top N , you can use sortByKey method and then use take method to get the top N value of your RDD ________________________________ mailforledkk From: Jeyaraj, Arockia R (Arockia)<mailto:arockia.r.jeya...@verizon.com> Date: 2014-04-09 21:55 To: user@spark.apache.org<mailto:user@spark.apache.org> Subject: To Ten RDD Hi , Can you any one tell how to get Top ten RDD by value? Thanks Arockia Raja
