HuangZhenQiu opened a new pull request, #20013:
URL: https://github.com/apache/hudi/pull/20013

   ### Describe the issue this Pull Request addresses
   
   
   HoodieCompactor.compact()/logCompact() and HoodieMergeHelper.runMerge() only 
closed their HoodieMergeHandle/HoodieAppendHandle on the success path. If 
doMerge()/doAppend()/executor.execute() threw (transient I/O error, OOM, schema 
issue, etc.), the handle's underlying HoodieFileWriter and its 
FSDataOutputStream were never closed. On cloud object stores such as GCS this 
leaks the resumable-upload stream (MediaHttpUploader) backing the write, since 
SimpleExecutor.shutdownNow() only releases the producer iterator and has no way 
to reach the consumer/mergeHandle to close it.
   
   Both call sites now close the handle on a best-effort basis when the 
merge/append/compaction step fails, before rethrowing the original failure. 
Close failures are attached as suppressed exceptions (compactor) or logged 
(merge helper) rather than masking the root cause.
   
   ### Summary and Changelog
   
   1. Fix the compactor and merge writer close operation
   2. Add test cases to cover the change
   
   ### Impact
   
   none
   
   ### Risk Level
   
   none
   
   ### Documentation Update
   
   none
   
   ### Contributor's checklist
   
   - [ ] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [ ] Enough context is provided in the sections above
   - [ ] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to