ewoodbury opened a new issue, #3378:
URL: https://github.com/apache/iceberg-rust/issues/3378
### Apache Iceberg Rust version
main
### Describe the bug
If none of a struct's requested children exist in a data file,
`get_arrow_projection_mask_with_field_ids` selects no leaf under the struct.
When another column is also projected, the struct is treated as absent and
reads as NULL for every row, including rows where the file has a non-null
struct. When the struct is the only projected column, the mask falls back to
`ProjectionMask::all()` and the struct validity comes out right.
Same mask behavior as #3374, but a different trigger: here the file has
field ids and the struct's children were all added after it was written, so
recursive name mapping (#1845) won't fix it. Iceberg Java's
`ParquetValueReaders.structReader` reads a column under the struct to get its
presence for this case.
Came up in review of #3255.
### To Reproduce
- File schema `{1: id int, 2: s struct<5: x int>}`, with `s` null in row 1
only.
- Table schema `{1: id int, 2: s struct<6: x int>}` (`x` dropped and
re-added with a new id).
- Projecting `[1, 2]` gives `s` validity `[false, false, false]`.
- Projecting `[2]` gives `[true, false, true]`.
### Expected behavior
`s` validity is `[true, false, true]` for both projections, with `x` null in
every row.
### Willingness to contribute
I would be willing to contribute a fix for this bug with guidance from the
Iceberg community
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
