cxzl25 opened a new pull request, #9706: URL: https://github.com/apache/paimon/pull/9706
### Purpose `SparkRemoveUnexistingFiles` creates a `TableCommit` on the executor inside `foreachPartition` but never closes it. `TableCommitImpl.close()` shuts down the underlying commit and the `maintainExecutor` thread pool, so every invocation of the procedure leaks a thread pool on a long-lived executor. Other call sites such as `PaimonPartitionManagement` already wrap the commit in try/finally. The intermediate RDD is also cached and never unpersisted. `buildRDD()` returned a child of the cached RDD to the caller, so the cache could only be released after the caller finished, which never happened. This change turns it into `execute()`, which collects the result inside the method and unpersists the cache in a `finally` block. Behaviour of the procedure is unchanged; only resource lifecycle is affected. ### Tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
