dhruv9b commented on issue #7664: URL: https://github.com/apache/paimon/issues/7664#issuecomment-5741159484
I investigated this issue on current master and was able to reproduce the behavior. With `write-buffer-spillable=true`, I found that: * `writer.close()` leaves the `paimon-io-*` spill directory. * Explicitly calling `ioManager.close()` removes the directory. * `IOManagerImpl.close()` delegates to `FileChannelManagerImpl`, which deletes the spill directory. I also traced the ownership/lifecycle pattern in the codebase. `withIOManager()` does not currently document ownership, but the existing Flink/Spark integrations consistently create the `IOManager`, pass it to the writer/reader, and close it themselves. Historical changes such as #2174 also explicitly added `IOManager.close()` at the components that create the IOManager. So it appears that the current design treats the `IOManager` as caller-owned rather than transferring ownership to `TableWrite`. Would the intended behavior for the Java API be to keep this ownership model and clarify the lifecycle in the `withIOManager()` Javadoc/examples, or should `TableWrite.close()` be responsible for closing an externally supplied `IOManager`? I have not made any production-code changes yet and wanted to confirm the intended ownership model before proceeding. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
