moomindani opened a new pull request, #2804: URL: https://github.com/apache/iceberg-rust/pull/2804
## Which issue does this PR close? - Closes #2803. Part of #2411 (Default Values). ## What changes are included in this PR? Per the spec, writers must use a column's `write-default` for columns they do not supply. DataFusion's insert planner consults `TableProvider::get_column_default` for columns omitted from an `INSERT` and falls back to `NULL`; `IcebergTableProvider` did not implement it, so tables with `write-default` values silently got `NULL`s. - `IcebergTableProvider` caches the schema's top-level `write-default` values as DataFusion expressions at construction and serves them via `get_column_default`. - Adds a `literal_to_scalar_value` conversion covering the primitive types (boolean, int, long, float, double, string, date, time, timestamp/timestamptz in µs and ns, decimal, binary, fixed, uuid); defaults with no scalar representation are skipped. The planner casts the expression to the target arrow type, so representation differences are reconciled downstream. - This mirrors iceberg-java's approach of delegating write-default application to the engine: its Spark integration exposes `write-default` as the engine column default (`TypeToSparkType`), and Spark materializes it at INSERT planning. - The static provider is read-only and keeps the trait default (no column defaults). ## Are these changes tested? Yes — an end-to-end test (`INSERT` omitting defaulted columns, then scanning to assert the defaults land and explicitly provided values win) plus unit tests for the literal conversion. The end-to-end test fails without the `get_column_default` implementation. `cargo test -p iceberg-datafusion --lib` (90 tests) and `cargo clippy -p iceberg-datafusion --lib --tests` pass locally. This pull request and its description were written by Claude Fable 5. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
