rangareddy commented on issue #17346: URL: https://github.com/apache/hudi/issues/17346#issuecomment-5351235970
This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-8764). **Findings: confirmed as a structural gap.** `hudi-common/src/main/java/org/apache/hudi/common/util/collection/BitCaskDiskMap.java` relies on exactly two cleanup mechanisms: - `writeOnlyFile.deleteOnExit();` at `:147` - the explicit `close()` at `:281-307`, which closes the file handles and calls `super.close()` Neither survives an abnormal exit. `File.deleteOnExit()` is implemented via a JVM shutdown hook, so it runs only on *normal* termination - it does nothing on `SIGKILL`, an OOM-killer kill, a YARN or Kubernetes container kill, or an executor lost mid-task. And `close()` obviously does not run if the process never unwinds. Those are exactly the cases your description calls out. The consequence is orphaned spill files accumulating in the executor's local directories, which on long-lived or frequently-preempted clusters is a real disk-space problem rather than a tidiness one. Since `deleteOnExit()` cannot be made robust here, the fix probably needs to be external to the process lifetime - for example writing spill files under a per-application/per-attempt directory that the engine already reclaims, or a startup sweep that removes stale spill directories belonging to dead application attempts. Worth deciding the approach in the ticket before coding, since it touches where the files are placed, not just when they are removed. Also relevant: the same "who cleans up on abnormal exit" question is what #17378 (HUDI-9181) gestures at for the metadata table reader. Worth checking whether they should be one piece of work. Keeping this open. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
