The following code fails with NullPointerException in RDD class on the
count function:
val rdd1 = sc.parallelize(1 to 10)
val rdd2 = sc.parallelize(11 to 20)
rdd1.map{ i =
rdd2.count
}
.foreach(println(_))
The same goes for any other action I am trying to perform
You cannot reference an RDD within a closure passed to another RDD. Your
code should instead look like this:
val rdd1 = sc.parallelize(1 to 10)
val rdd2 = sc.parallelize(11 to 20)
val rdd2Count = rdd2.count
rdd1.map{ i =
rdd2Count
}
.foreach(println(_))
You
