JunRuiLee commented on PR #563: URL: https://github.com/apache/paimon-rust/pull/563#issuecomment-5045261119
Thanks @JingsongLi for the thorough review and for verifying the build/package/CI locally. All four points are addressed in `2cc6c68`: **1. Re-export the core I/O types** — `paimon::ftindex` now re-exports `SeekRead`, `ReadRequest`, and `SliceReader` (plus `FullTextArchiveReader`, `FullTextHits`, and the new `BytesReader`), so a consumer can implement `SeekRead` or name the reader while depending only on `paimon`. Added `test_from_seek_read_opens_archive_directly`, which exercises the generic constructor directly (bypassing `from_input_file`) — the entry point #571's remote-`FileIO` path uses. **2. Avoid consuming the include bitmap** — `search_with_include` now takes `&RoaringTreemap`; it only serializes the bitmap, so callers reusing an allow-list across archives no longer clone. It also returns an empty `FullTextHits` immediately when the allow-list is empty, skipping the query entirely. **3. Memory claim in `from_input_file`** — went with the stronger fix rather than only correcting the doc: added a `Bytes`-backed `BytesReader` and switched `from_input_file` to hold the `Bytes` from `input.read()` directly, so there is no longer a second `to_vec()` buffer — peak is genuinely ~one archive now. Doc updated to match, and `from_seek_read` remains the path for large/remote archives that should not be fully buffered. **4. PR description** — updated to reflect the crates.io `0.1.0` dependency, the generic `SeekRead` API + re-exports, the ready (non-draft) state, and that `cargo package -p paimon` succeeds. Verification after the change: `cargo test -p paimon --features fulltext` (the four `ftindex::reader` tests + full suite), `cargo clippy -p paimon --lib --tests --features fulltext -- -D warnings`, default build, minimal `fulltext + storage-memory` build, `cargo package -p paimon`, and `cargo build -p paimon-datafusion` all pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
