c-thiel opened a new issue, #3608: URL: https://github.com/apache/iceberg-rust/issues/3608
### Apache Iceberg Rust version main @ 1ee5c72a7 ### Describe the bug `reader/pipeline.rs` decides whether a Parquet file has field ids from the first top-level field only (`missing_field_ids`). If that field has no id and there is no name mapping, every top-level column gets a positional fallback id, overwriting the ids the other columns carry. A column whose embedded id differs from its 1-based position then silently reads as NULL, or as another column's values. The code comment says this corresponds to Java's `ParquetSchemaUtil.hasIds()`. That check is true if any field at any depth has an id. ### To Reproduce File `optional int32 junk; optional int32 c = 4;` (no id on `junk`) with `c = [7, 8]`. Table schema `optional 4: c int`, projecting `[4]`, no name mapping. Actual: `c = [null, null]`. With `c = 1` and `junk = [100, 200]`, the scan returns `junk`'s values `[100, 200]` for `c`. ### Expected behavior `c = [7, 8]`, as iceberg-java returns for both files (run on 1.9.1; `ReadConf` and `ParquetSchemaUtil.HasIds` are unchanged through 1.12.0). Treat a file as having no field ids only if no field at any depth has one. ### Willingness to contribute I can contribute a fix for this bug independently -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
