checkpoint seems to be just add a CheckPoint mark? You need an action after marked it. I have tried it with success:)
newRdd = oldRdd.map(myFun).persist(myStorageLevel) newRdd.checkpoint // <<checkpoint here newRdd.isCheckpointed // false here newRdd.foreach(x => {}) // Force evaluation newRdd.isCheckpointed // true here oldRdd.unpersist(true) ~~~~~~~~ If you have new broadcast object for each step of iteration, broadcast will eat up all of the memory. You may need to set "spark.cleaner.ttl" to a small enough value. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Incredible-slow-iterative-computation-tp4204p5407.html Sent from the Apache Spark User List mailing list archive at Nabble.com.