Since disk here is HDFS & HDFS only supports appending of files, i bet that RDD is rewritten. Never tried this though.
Mayur Rustagi Ph: +919632149971 h <https://twitter.com/mayur_rustagi>ttp://www.sigmoidanalytics.com https://twitter.com/mayur_rustagi On Thu, Feb 20, 2014 at 10:41 AM, Adrian Mocanu <amoc...@verticalscope.com>wrote: > Hi > > When saving an RDD to disk, if the spark node doing the saving crashes, is > the entire file rewritten to disk or can spark figure out where it left off > upon RDD recomputation? > > > > -Adrian > > >
