zhengruifeng opened a new pull request, #57754: URL: https://github.com/apache/spark/pull/57754
### What changes were proposed in this pull request? This PR adds a live ML cache status column to the session table in the Spark Connect UI. For active sessions, the column shows the number of objects currently in memory and the current and maximum in-memory and total cache sizes. It also reports when memory control is disabled or the ML cache has not been used. Closed sessions and History Server views show `N/A` because their live cache state is unavailable. The UI reads the cache status without updating the session's last-access time or initializing an unused ML cache. ### Why are the changes needed? Spark Connect ML models are cached per session, but operators currently cannot inspect cache utilization in the Spark UI. Showing the live status makes it easier to understand model retention, memory pressure, and disk offloading for each session. ### Does this PR introduce _any_ user-facing change? Yes. The Spark Connect session table has a new `ML Cache` column showing live per-session cache utilization. ### How was this patch tested? Added unit coverage for ML cache status collection and Connect UI rendering. Tests have not been run yet; this is a draft PR. `git diff --check` passes. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Codex (GPT-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
