jja725 opened a new issue, #637: URL: https://github.com/apache/hudi-rs/issues/637
## Summary The C++ FFI's slice-read entry points in `cpp/src/lib.rs` — `read_file_slice` and `read_file_slice_from_paths` — construct `ReadOptions::new()` internally and pass that to the inner reader. As a result, any read options the caller might want (column projection, filters, read-optimized mode, batch size) never reach the read. The Rust core already supports these: `FileGroupReader::read_file_slice_from_paths` honors `options.projection` and filters and applies them to the merged result. Only the C++ shim drops them. ## Impact for engine integrations Engines embedding the C++ library (e.g. Presto native / Velox — see apache/hudi#18308) get no column pruning or predicate pushdown through the FFI. The merged batch materializes all columns for the slice even when the query needs a few, which is both slower and heavier on memory (a MOR slice with logs is merged in memory and returned as a single batch). ## Proposal Widen the C++ slice-read entry points to accept read options and forward them to the inner reader. A simple, `cxx`-friendly shape is a `Vec<String>` of `key=value` options (consistent with `new_file_group_reader_with_options`), with a reserved `projection=col1,col2` key mapped to `ReadOptions::with_projection`; other keys map to `with_hudi_option`. A dedicated projection parameter would also work. This enables column projection immediately and leaves a clear path to filter pushdown later, without further FFI signature changes. ## Notes Pairs with the panic-safety hardening tracked separately. Both are small, isolated changes to `cpp/src/lib.rs` and are needed for a usable query-engine integration. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
