dragonknightk opened a new pull request, #11182:
URL: https://github.com/apache/paimon/pull/11182

   ### Purpose
   
   Fixes #11111
   
   `FileStoreCommitImpl.abort` now deletes the files of an aborted write 
concurrently, bounded by `file-operation.thread-num`. Before this change, each 
delete ran on the caller thread one after another, so abort time grew with the 
file count times the per-delete latency.
   
   - Abort first collects the paths with the existing rules, then deletes them 
as one batch. The set of files owned by the aborted write is unchanged, 
including the #10582 rule that keeps files reused by compaction.
   - The deletes run on the shared `FileOperationThreadPool`, the same pool and 
option that snapshot expiration (#8241) uses. No option or pool is added. Each 
delete runs with the caller's context class loader, as the sequential abort did.
   - Every path gets a delete attempt, and the caller blocks until all attempts 
have finished, including when a delete fails or the caller is interrupted.
   - Each delete still goes through `FileIO.deleteQuietly`, so an IOException 
is logged and does not throw. Any other exception is rethrown as is after all 
attempts, with later failures suppressed into the first in input order.
   - An interrupt on the caller thread does not cancel the cleanup. Abort keeps 
waiting for all attempts and restores the interrupt flag before returning or 
throwing.
   
   ### Tests
   
   - With `file-operation.thread-num = 4` and a FileIO that delays each 
operation, the deletes of one abort overlap, with 2 to 4 in flight. On the code 
before this change at most one delete is in flight, so this test fails there. 
Each delete sees the caller's context class loader, and the pool threads keep 
their own context class loaders after abort.
   - All files owned by the aborted write are deleted, and a file reused by 
compaction stays.
   - When one delete fails with an IOException, abort does not throw and still 
deletes every other file.
   - When the caller is interrupted before abort and 
`file-operation.thread-num` is below the shared pool size, abort still deletes 
every file and the interrupt flag stays set.
   - When the caller is interrupted while abort waits for the deletes, abort 
still waits for and deletes every file and the interrupt flag stays set.
   - In the same interrupted case with two deletes failing with unchecked 
exceptions, abort deletes every other file, throws the first failure in input 
order with its original type, and carries the later one as its only suppressed 
exception. The interrupt flag stays set.
   
   ## AI Disclosure
   
   Human-led contribution. Grok Bot, working for dragonknightk, assisted with 
this change, including the code, tests, and description. All submitted content 
was reviewed by a human before submission.
   
   (๑•̀ㅂ•́)و✧
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to