NoahKusaba opened a new pull request, #14: URL: https://github.com/apache/datafusion-iceberg/pull/14
## Which issue does this PR close? - Closes #13. - 1/5 of planned PR's that will close the Ballista-Iceberg integration. Ported from https://github.com/apache/iceberg-rust/pull/3001, which was opened against `apache/iceberg-rust` before the DataFusion integration moved here. ## What changes are included in this PR? `PartitionExpr` wraps a `PartitionValueCalculator`, which is a live object with no serializable form. Once the expression is built, nothing on it says which partition spec and schema produced it, so a caller holding the expression cannot rebuild an equivalent one. Retain both inputs on the expression and expose them: - `try_new(partition_spec, table_schema)` replaces `new(calculator, spec)` and builds the calculator itself, so the two inputs and the calculator cannot drift apart. - `partition_spec()` and `table_schema()` read them back. Both are self-contained iceberg spec types, so a distributed engine can serialize them and rebuild the expression on a worker with `try_new`. `project_with_partition` is unchanged from the outside; it now hands the two inputs to `try_new` instead of constructing the calculator first. ## Are these changes tested? - `test_partition_expr_rebuilds_from_its_retained_parts`: reads `partition_spec()` and `table_schema()` off a `PartitionExpr`, builds a second one from them via `try_new`, and asserts both evaluate a `RecordBatch` to the same array. The existing project tests cover the refactor itself, as it now constructs through `try_new`, and they still pass unchanged. `cargo fmt --all -- --check`, `cargo clippy --workspace --locked --all-targets -- -D warnings` and `cargo test --workspace --locked` all pass locally. ## AI Disclosure - Helped write the test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
