kete1987 commented on code in PR #56234:
URL: https://github.com/apache/spark/pull/56234#discussion_r3337171275
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDB.scala:
##########
@@ -2133,6 +2133,15 @@ class RocksDB(
nativeStats.close()
rocksDbOptions.close()
dbLogger.close()
+ // In unbounded memory mode each RocksDB instance owns its LRUCache.
Without explicit
+ // close() the native C++ cache object is only freed when the JVM GC
finalizes the Java
+ // wrapper -- which rarely happens under low heap pressure. Closing
explicitly here
+ // ensures native memory is reclaimed deterministically when the
instance is released.
+ // In bounded mode the cache is a shared singleton managed by
RocksDBMemoryManager
+ // and must not be closed here.
+ if (!conf.boundedMemoryUsage && lruCache != null) {
Review Comment:
Yes, exactly. In unbounded mode each RocksDB instance owns its LRUCache
exclusively, so it's correct to close it deterministically on close(). Without
this, the native C++ object is only freed when the JVM GC finalizes the Java
wrapper, which is non-deterministic and can be significantly delayed under low
heap pressure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
