jackylee-ch opened a new pull request, #10164: URL: https://github.com/apache/paimon/pull/10164
### Purpose `SystemReadBuilder.with_filter` stores and forwards a predicate, but `SystemTableRead._materialise` raised `NotImplementedError` for **any** predicate. So `with_filter(...)` on a system table (`db.t$snapshots`, `db.t$files`, ...) always failed, and layers that push filters down to pypaimon (Daft / Ray) hit this on every system table. This applies the predicate to the materialised PyArrow table for the methods that convert to a safe Arrow row filter (comparisons, null checks, `is_in`), reusing `predicate_supports_arrow_filter` from the data-read path. The filter runs before projection, so a predicate may reference columns that are not projected. String-match predicates (`starts_with` / `ends_with` / `contains` / `like`) are not safe as final Arrow row filters and still raise a clear error. ### Tests `pypaimon/tests/system/system_table_test.py`: added `SystemTableFilterTest` covering equal / greater-than / is-in filtering, filtering on a column that is not in the projection, and the string-match method still raising `NotImplementedError`. ### API and Format No public API or format change; `with_filter` was already exposed on the system-table read builder. Written with Claude Code; verification is mine. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
