LuciferYang opened a new pull request, #10943: URL: https://github.com/apache/paimon/pull/10943
### Purpose close #10941 `BatchReadTagCleanupListener` deleted every pending batch read protection tag whenever a query finished, so one query finishing removed the tags of other queries that were still reading. The listener is now a `SparkListener`, added once per `SparkContext`: - A scan planned inside a running SQL execution registers its tag under that execution id, and the tag is deleted when that execution ends. - A scan planned outside a running execution (`Dataset.rdd` on Spark 3, `Dataset.cache`) keeps the old behavior: its tag is deleted when the next execution ends. Only executions that end after the scan was planned count, so a late end event of an earlier execution does not delete it. - An execution id that is no longer running at planning time is treated as no id, and ids whose executions ended without their end event reaching the listener are dropped on the next end event. TTL expiration still cleans up any tag this misses. ### Tests New tests in `BatchReadTagProtectionTest`: - "a query ending does not clean the protection tag of a running query": another query ends while a scan runs inside a nested SQL execution; its tag must survive until that execution ends. Fails on master. - "a scan planned with the id of an ended execution is cleaned by the next query". - "a late end event of an earlier execution does not clean the tag of a later scan": holds the listener bus so the end event of `SELECT 1` arrives after a scan is planned outside an execution. - "the protection tag of a cached scan is cleaned after the cache is built". ### API and Format no ### Documentation no -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
