Github user jose-torres commented on the issue:
https://github.com/apache/spark/pull/20710
Partitions are a better example than task attempts, but it's still roughly
the same idea. Data source writers need to be able to reason about what
progress they've made, which is impossible in the streaming case if each epoch
is its own disconnected query.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
