voonhous opened a new pull request, #19936: URL: https://github.com/apache/hudi/pull/19936
### Describe the issue this Pull Request addresses `InLineFsDataInputStream` reads an HFile embedded in a log file. It wraps the outer file's `FSDataInputStream` in an offset-adjusting `InputStream` and passes that to its super constructor. Neither class overrides `close()`, so closing an inline stream ends at `InputStream.close()`, a no-op, and the outer file's stream stays open until garbage collection. Every inline HFile read of a metadata table log block leaks a file handle this way. Found by TestStreamingSource on #19932, where Spark's `DebugFilesystem` reported the outer streams as leaked once it was the JVM's cached local filesystem. On master the check never sees Hudi's reads, because the SQL suites cache a plain LocalFileSystem first. ### Summary and Changelog - `InLineFsDataInputStream.close()` closes the wrapped stream and then the outer stream. - Unit test in TestInLineFileSystem that wraps the outer stream in a counting `FSDataInputStream` and asserts it is closed exactly once. ### Impact Metadata table reads through the inline filesystem release their file handles when the reader closes instead of at GC. ### Risk Level low. One `close` override; the outer stream is owned by this object. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
