HeartSaVioR opened a new pull request, #52903: URL: https://github.com/apache/spark/pull/52903
### What changes were proposed in this pull request? This PR proposes to fix the bug of missing close() on underlying InputPartition when LowLatencyReaderWrap.close() is called. ### Why are the changes needed? Not closing the underlying InputPartition could leak resource; e.g. Kafka consumer is not returned to the pool, which ends up with destroying the purpose of connection pool and creating Kafka consumer instances every batch. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? A new UT for Kafka rather than general one, since Kafka data source has an internal metric to provide the necessary information for validation. ### Was this patch authored or co-authored using generative AI tooling? No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
