Just wanted to add a comment to the Jira ticket but I don't think I have permission to do so, so answering here instead. I am encountering the same issue with a stackOverflow Exception. I would like to point out that there is a localCheckpoint <https://jaceklaskowski.gitbooks.io/mastering-apache-spark/spark-rdd-checkpointing.html> method which does not require HDFS to be installed. We could use this instead of Checkpoint to cut down the lineage.
-- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
