Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/21500
@aalobaidi
When starting batch, latest version state is being read to start a new
version of state. If the state should be restored from snapshot as well as
delta files, it will incur huge latency on restoring.
#21506 logs messages when loading state requires dealing with (remote)
filesystem. That's why I suggest to merge my patch and run your case again.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
