JunRuiLee opened a new issue, #413: URL: https://github.com/apache/paimon-rust/issues/413
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon-rust/issues) and found nothing similar. ### Motivation PyPaimon has two read paths today: - **SQL** (`SQLContext.sql`) — already runs on the Rust DataFusion engine. - **DataFrame** (`ReadBuilder → Split → TableRead.to_arrow/to_pandas/to_ray`) — still **pure Python**, even though the Rust core already implements the same model in `crates/paimon/src/table/read_builder.rs`. It's just not exposed through `bindings/python` (`PyTable` only has `identifier/location/schema`). Goal: expose the existing Rust read API to Python so the DataFrame read path can run on Rust, with the public Python API unchanged. Write path is out of scope. ## Scope (incremental PRs) - **PR 1** — Expose `ReadBuilder` + scan: `new_read_builder()`, `with_filter/ with_projection/with_limit`, and `new_scan().plan()` returning serializable splits. - **PR 2** — Expose split → Arrow read: `new_read().read(splits)` returning a `RecordBatchReader` backed by Rust `TableRead.read()` (Arrow FFI, zero-copy). - **PR 3** (in `apache/paimon`, `[python]`) — Switch PyPaimon's `to_arrow/ to_pandas/to_ray` to use the Rust reader internally. PR 1–2 land here; PR 3 lands in the main repo once bindings are released. ### Solution _No response_ ### Anything else? _No response_ ### Willingness to contribute - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
