ewoodbury opened a new issue, #3261:
URL: https://github.com/apache/iceberg-rust/issues/3261
`RecordBatchTransformer` null-fills a nested field that is missing from an
older data file. If that field has an `initial-default`, it returns
`FeatureUnsupported` instead of applying the default:
```
initial-default of nested field {name} is not supported
```
Top-level fields already apply `initial-default` through
`ColumnSource::Add`. Nested fields do not.
The spec tracks defaults on each field, including fields inside a struct. A
parent struct's default must not embed its children's defaults. The effective
value is built by applying each field's own default.
From the spec's `point` example: `point.y` has `initial-default` 0, and an
older file has `{"x": 3}`. The read should return `{"x": 3, "y": 0}`, not null
for `y`.
https://iceberg.apache.org/spec/#default-values
This shows up on v3 tables when a nested field was added with a non-null
`initial-default` and older files are still in the snapshot. Those reads fail
until this is implemented. Failing is preferable to silently returning null.
Out of scope for #3255. That PR should keep the error and point at this
issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
