discivigour opened a new pull request, #10977:
URL: https://github.com/apache/paimon/pull/10977

   ### Purpose
   
   A lookup merge can spill records for the current key to local disk. Moving 
to the next key resets the merge state and releases those files. If the reader 
closes while processing a key, or immediately after it returns the last key, 
the spill files can survive until the IO manager shuts down.
   
   Reset the merge function wrapper when closing both the loser-tree and 
min-heap readers. If closing an input fails, still attempt the reset and 
preserve the input exception, with any reset failure attached as a suppressed 
exception.
   
   This fix was split out of #10867 because existing lookup merge readers can 
also leave spill files behind.
   
   ### Tests
   
   `SortMergeReaderCleanupTest` reproduces the bug with both sort engines. It 
forces a real spill, closes the reader while the IO manager is still open, and 
verifies that the files have been removed and the returned value remains 
readable. A second case checks the exception and cleanup behavior when closing 
the input fails. All four parameterized cases failed before the fix and passed 
with it.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to