Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21067
> We don't have a solid story for checkpointing streaming computation right
now, and even if we did, you'll certainly lose all progress from batch jobs.
Should probably clarify re: streaming - we don't do any Kubernetes-specific
actions (e.g. Persistent Volumes) to do Streaming checkpointing. But anything
built-in to Spark should work, such as DFS checkpointing - barring anything
that requires using the pod's local disk.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
