Github user TomaszGaweda commented on the issue:
https://github.com/apache/spark/pull/21500
@HeartSaVioR IMHO we should consider new state provider such as RocksDB,
like Flink and Databricks Delta did. It is not a direct fix, but will improve
latency and memory consumption, maybe additional management on Spark side won't
be required
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
