[ https://issues.apache.org/jira/browse/SPARK-718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14142506#comment-14142506 ]
Matthew Farrellee commented on SPARK-718: ----------------------------------------- Spark simply does not support nesting RDDs in this fashion. you'll get a more prompt response and information with the user list, see http://spark.apache.org/community.html. i'm going to close this issue, but if you want feel free to re-open it. > NPE when performing action during transformation > ------------------------------------------------ > > Key: SPARK-718 > URL: https://issues.apache.org/jira/browse/SPARK-718 > Project: Spark > Issue Type: Bug > Affects Versions: 0.7.0 > Reporter: Krzywicki > > Running the spark shell: > The following code fails with a NPE when trying to collect the resulting RDD: > {code:java} > val data = sc.parallelize(1 to 10) > data.map(i => data.count).collect > {code} > {code:java} > ERROR local.LocalScheduler: Exception in task 0 > java.lang.NullPointerException > at spark.RDD.count(RDD.scala:490) > at > $line16.$read$$iwC$$iwC$$iwC$$iwC$$anonfun$1.apply$mcJI$sp(<console>:15) > at $line16.$read$$iwC$$iwC$$iwC$$iwC$$anonfun$1.apply(<console>:15) > at $line16.$read$$iwC$$iwC$$iwC$$iwC$$anonfun$1.apply(<console>:15) > at scala.collection.Iterator$$anon$19.next(Iterator.scala:401) > at scala.collection.Iterator$class.foreach(Iterator.scala:772) > at scala.collection.Iterator$$anon$19.foreach(Iterator.scala:399) > at > scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48) > at > scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:102) > at > scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:250) > at scala.collection.Iterator$$anon$19.toBuffer(Iterator.scala:399) > at > scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:237) > at scala.collection.Iterator$$anon$19.toArray(Iterator.scala:399) > at spark.RDD$$anonfun$1.apply(RDD.scala:389) > at spark.RDD$$anonfun$1.apply(RDD.scala:389) > at spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:610) > at spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:610) > at spark.scheduler.ResultTask.run(ResultTask.scala:76) > at > spark.scheduler.local.LocalScheduler.runTask$1(LocalScheduler.scala:74) > at > spark.scheduler.local.LocalScheduler$$anon$1.run(LocalScheduler.scala:50) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:722) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org