micheal-o commented on code in PR #50512:
URL: https://github.com/apache/spark/pull/50512#discussion_r2034331452
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDB.scala:
##########
@@ -885,6 +885,10 @@ class RocksDB(
val (dfsFileSuffix, immutableFileMapping) =
rocksDBFileMapping.createSnapshotFileMapping(
fileManager, checkpointDir, version)
+ logInfo(log"RocksDB file mapping after creating snapshot file mapping
for version " +
Review Comment:
@HeartSaVioR this is logged only when we create a snapshot (e.g. 1 per 10
batches), hence i don't expect it to be noisy or a lot. The content is just
list of local SST files and what DFS files they are mapped to. From past
experience this is also not typically a lot. Maybe, on average, should be less
than 10 - 20 SST files.
This would really make debugging this type of issue easier and faster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
