anishshri-db opened a new pull request, #46233:
URL: https://github.com/apache/spark/pull/46233
### What changes were proposed in this pull request?
Improve logging around snapshot creation and adding/removing entries from
state cache map
### Why are the changes needed?
Sometimes we have insufficient context while debugging issues related to
HDFS provider snapshot creation as well as entries kept in `loadedMaps`. This
change tries to improve that
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Manually verified the logs:
```
sql/core/target/unit-tests.log:3128298:14:02:56.329
pool-1-thread-1-ScalaTest-running-StateStoreSuite INFO
HDFSBackedStateStoreProvider: Trying to add version=1 to state cache map with
current_size=0 and max_versions_to_retain_in_memory=2
sql/core/target/unit-tests.log:3128302:14:02:56.329
pool-1-thread-1-ScalaTest-running-StateStoreSuite INFO
HDFSBackedStateStoreProvider: Trying to add version=2 to state cache map with
current_size=1 and last_loaded_version=1 and max_versions_to_retain_in_memory=2
```
### Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
