Github user ferdonline commented on the issue:
https://github.com/apache/spark/pull/9428
That's the reason why I want to checkpoint when they are first calculated.
Further transformations use these results several times. Of course it's not a
problem per se to calculate twice for the checkpoint, but doing so for 1+TB of
data is nonsense and I can't cache.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
