Re: persist versus checkpoint

2016-04-30 Thread Holden Karau
They are different, also this might be better suited for the user list. Persist by default will cache in memory on one machine, although you can specify a different storage level. Checkpoint on the other hand will write out to a persistent store and get rid of the dependency graph used to compute t

persist versus checkpoint

2016-04-30 Thread Renyi Xiong
Hi, Is RDD.persist equivalent to RDD.checkpoint If they save same number of copies (say 3) to disk? (I assume persist saves copies on different machines ?) thanks, Renyi.