You can use cache or persist.
On Tue, Oct 18, 2016 at 10:11 AM, Yang <teddyyyy...@gmail.com> wrote:
> I'm trying out 2.0, and ran a long job with 10 stages, in spark-shell
> it seems that after all 10 finished successfully, if I run the last, or
> the 9th again,
> spark reruns all the previous stages from scratch, instead of utilizing
> the partial results.
> this is quite serious since I can't experiment while making small changes
> to the code.
> any idea what part of the spark framework might have caused this ?