Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/21700#discussion_r201793040
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateStoreSuite.scala
---
@@ -64,6 +64,63 @@ class StateStoreSuite extends
StateStoreSuiteBase[HDFSBackedStateStoreProvider]
require(!StateStore.isMaintenanceRunning)
}
+ test("retaining only latest configured size of versions in memory") {
--- End diff --
Sorry I didn't catch this earlier. We should ideally have tests that
directly validate the specific behaviors we're documenting in the conf:
* '2' will read from cache in the direct failure case
* '1' will read from cache in the happy path but not if there's a failure
* '0' will never populate or read from the cache
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
