vinothchandar opened a new pull request, #17997:
URL: https://github.com/apache/hudi/pull/17997
### Describe the issue this Pull Request addresses
InMemoryFileSystem is unused now, since we pass in the outputstream object
for both parquet and hfile writers. In current shape, it has also unresolved
issues to support executors with num_cores >1. Dropping to avoid any accidents.
### Summary and Changelog
Deleted files (2 files, 267 lines):
-
hudi-hadoop-common/src/main/java/org/apache/hudi/hadoop/fs/inline/InMemoryFileSystem.java
- Dead production code
-
hudi-hadoop-common/src/test/java/org/apache/hudi/common/fs/inline/TestInMemoryFileSystem.java
- Tests for dead code
Refactored files (2 files):
- TestInLineFileSystemWithHFileReader.java - Now uses
ByteArrayOutputStream instead of InMemoryFileSystem to generate HFile bytes
- TestParquetInLining.java - Now writes Parquet to a temp file on disk,
reads the bytes, then proceeds with the InLineFileSystem test
Modified file:
- FileSystemTestUtils.java - Removed InMemoryFileSystem import and
getRandomOuterInMemPath() method
Key Changes in Test Files
1. TestInLineFileSystemWithHFileReader: The HFileWriter now writes to a
ByteArrayOutputStream directly (line 100-106), then embeds those bytes in the
outer file. This preserves the test of reading HFile content via
InLineFileSystem.
2. TestParquetInLining: Since ParquetWriter requires a Path, it now writes
to a temp file on disk first (line 90-98), reads the bytes from that file (line
101), then embeds them in the outer file. The temp file is cleaned up in
teardown().
Verification
- No references to org.apache.hudi.hadoop.fs.inline.InMemoryFileSystem
remain
- No references to inmemfs scheme remain
- Both InLineFileSystem test files are preserved and refactored
### Impact
None. Dead code.
### Risk Level
None
### Documentation Update
none
### Contributor's checklist
- [ ] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Enough context is provided in the sections above
- [ ] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
