JunRuiLee opened a new pull request, #419:
URL: https://github.com/apache/paimon-rust/pull/419
### Purpose
Second PR of exposing PyPaimon's DataFrame read path to Rust (follows #415).
Refs #413.
Adds `ReadBuilder.with_filter(predicate: dict)` — converts a lightweight
dict predicate into a Rust `Predicate` (resolving fields against the table
schema) and pushes it into scan planning so `plan()` can prune splits.
```python
table.new_read_builder().with_filter(
{"method": "equal", "field": "id", "literals": [1]}
).new_scan().plan()
```
Supports the directly-translatable subset:
`equal/notEqual/lessThan/lessOrEqual/greaterThan/greaterOrEqual/isNull/isNotNull/in/notIn/and/or`.
Out of scope (later PRs): `read(splits) → Arrow` data reading, pypaimon-side
wiring.
### Notes
- **Schema is authoritative**: any `index`/`data_type` in the dict is
ignored; fields resolve by name against the current table schema.
- **No partial pushdown**: if any node (incl. a compound child) is
unsupported, the whole `with_filter` fails — nothing is partially pushed.
- Unsupported operators (`like/startsWith/endsWith/contains/not`) and
unsupported literal types (`Date/Time/Timestamp/Decimal/Bytes`, complex) →
`NotImplementedError`. Unknown field / wrong literal count / `None` misuse /
type mismatch / empty compound children → `ValueError`.
- Filter pushdown is conservative: successful conversion does not guarantee
split-count reduction for every predicate; it enables read-side pruning where
supported. Exact row filtering is left to a residual layer.
### Tests
`bindings/python/tests/test_read.py` — filter conversion
(eq/and/or/in/isNull/bool), partition pruning, and error paths (unsupported
op/type, unknown field, wrong count, empty children).
### Note on CI
CI will be red until #418 merges: current `main` does not compile
(`DataType::Vector` not covered in `arrow/format/row.rs`, unrelated to this
PR). This PR is built on `main` and will go green once #418 lands; happy to
rebase then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
