neilramaswamy opened a new pull request, #48853: URL: https://github.com/apache/spark/pull/48853
### What changes were proposed in this pull request? This PR ensures that the secondary indexes that state variables with TTL use are at most the size of the corresponding state variable's primary index. This change will eliminate unnecessary work done during the cleanup of stateful variables with TTL. ### Why are the changes needed? [Long explanation incoming.] ### How should reviewers review this PR? <TODO> ### Does this PR introduce _any_ user-facing change? No, but it is a format difference in the way TWS represents its internal state. However, since TWS is currently `private[sql]` and not publicly available, this is not an issue. ### How was this patch tested? - Existing UTs have been modified to conform with this new behavior. - New UTs added to verify that the new indices we added ### Was this patch authored or co-authored using generative AI tooling? Generated-by: GitHub Copilot -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
