Hi all The description about this Bug submitted by Matei is as following
The tipping point seems to be around 50. We should fix this by checkpointing the RDDs every 10-20 iterations to break the lineage chain, but checkpointing currently requires HDFS installed, which not all users will have. We might also be able to fix DAGScheduler to not be recursive. regards, Andrew