dragonknightk commented on issue #11111: URL: https://github.com/apache/paimon/issues/11111#issuecomment-6905579750
Hi @wangzhigang1999, thanks for the detailed report and measurements. I'd like to work on this, here is my plan: - `FileStoreCommitImpl.abort` first collects the paths with the existing rules, then deletes them as one batch. The set of deleted files stays the same, including keeping files reused by compaction. - The deletes run on the shared `FileOperationThreadPool`, bounded by `file-operation.thread-num`, the same pool and option that snapshot expiration uses. No new option or pool. - The caller still waits until every path has had a delete attempt. Each delete still goes through `deleteQuietly`, so an IOException is only logged as before. Any other exception is rethrown with its original type after all attempts, with later failures suppressed into the first. - An interrupt on the caller thread does not cancel the cleanup. Abort keeps waiting and restores the interrupt flag at the end. Each delete runs with the caller's context class loader, as the sequential abort did. With a FileIO that delays each operation, the deletes of one abort now overlap up to the thread limit, while on the current code at most one is in flight. The change and tests are ready in my fork. Could this issue be assigned to me? ʕ•ᴥ•ʔ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
