dragonknightk commented on issue #11111:
URL: https://github.com/apache/paimon/issues/11111#issuecomment-6905579750

   Hi @wangzhigang1999, thanks for the detailed report and measurements. I'd 
like to work on this, here is my plan:
   
   - `FileStoreCommitImpl.abort` first collects the paths with the existing 
rules, then deletes them as one batch. The set of deleted files stays the same, 
including keeping files reused by compaction.
   - The deletes run on the shared `FileOperationThreadPool`, bounded by 
`file-operation.thread-num`, the same pool and option that snapshot expiration 
uses. No new option or pool.
   - The caller still waits until every path has had a delete attempt. Each 
delete still goes through `deleteQuietly`, so an IOException is only logged as 
before. Any other exception is rethrown with its original type after all 
attempts, with later failures suppressed into the first.
   - An interrupt on the caller thread does not cancel the cleanup. Abort keeps 
waiting and restores the interrupt flag at the end. Each delete runs with the 
caller's context class loader, as the sequential abort did.
   
   With a FileIO that delays each operation, the deletes of one abort now 
overlap up to the thread limit, while on the current code at most one is in 
flight. The change and tests are ready in my fork. Could this issue be assigned 
to me? ʕ•ᴥ•ʔ
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to