Felicuss opened a new pull request, #975: URL: https://github.com/apache/hudi-rs/pull/975
Metadata HFiles currently ignore `hoodie.hfile.compression.algorithm` and are always written uncompressed. Add block-level GZIP writing and propagate the data table's resolved `GZ`/`NONE` option to metadata bootstrap, log blocks, and compaction output for all supported metadata partitions. Existing writer APIs and the `NONE` default are preserved. Unsupported codecs fail before writing; mixed compressed and uncompressed files remain readable. Existing files are only rewritten when their slices are compacted. Validation on macOS arm64 with Rust 1.94.1: - `make check-rust` and `make check-python`. - `make test-rust`: 2175 passed, 20 ignored. - Core without default features: 1689 passed, 4 ignored. - `make develop` and `make test-python`: 174 passed. - Native JVM HFile interoperability in both directions, including full scans and point lookups; NONE output is byte-identical to the baseline writer on generated fixtures. - GZIP Spark compaction interoperability passed. Eight additional Spark parity scenarios passed; the broader run was interrupted during the index-lookup scenario, leaving that scenario, SQL DML, and the existing NONE compaction scenario without a result. The user requested proceeding without further test runs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
