JunRuiLee opened a new pull request, #428: URL: https://github.com/apache/paimon-rust/pull/428
### Purpose Third PR of exposing PyPaimon's DataFrame read path to Rust (follows #415 scan/plan/splits and #419 `with_filter`). Refs #413. Adds the data-reading step both prior PRs deferred: `ReadBuilder.new_read().read(splits) → List[pyarrow.RecordBatch]`. ```python b = table.new_read_builder().with_projection(["id"]) splits = b.new_scan().plan().splits() batches = b.new_read().read(splits) # -> List[pyarrow.RecordBatch] ``` Splits are picklable (from #415), so they can be planned in one process and read in another — groundwork for Ray distributed reads. ### Notes - **Shared config**: extracts `apply_read_config` (projection/limit/filter) so `new_scan().plan()` and `new_read().read()` apply the same pushdown consistently. Pure refactor, no behavior change to scan. - **Config snapshot**: `new_read()` snapshots the builder's projection/limit/filter, so mutating the builder afterward does not affect an already-created read. - **Projection now applies**: projection was a planning no-op in #415; it takes effect here on the read path. - **Eager collection**: `read()` collects the full Arrow stream into a `List[RecordBatch]`. - **Errors**: a non-iterable argument or a non-`Split` element raises `TypeError`; empty splits return `[]` without entering the runtime. Off-GIL read via `py.detach` + runtime `block_on`, matching the existing `plan()` pattern. Out of scope (later PRs): pypaimon-side wiring. ### Tests `bindings/python/tests/test_read.py` — 7 new tests: expected rows, empty splits, non-`Split` `TypeError`, pickled-split round-trip, projection applied, config-snapshot isolation, and a `with_filter` read smoke test. Full file 33/33 passing; `cargo clippy` clean. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
