joyhaldar opened a new issue, #18543: URL: https://github.com/apache/iceberg/issues/18543
### Apache Iceberg version main (development) ### Query engine Spark ### Please describe the bug 🐞 When reading an ORC file with the vectorized Spark reader, the `_row_id` metadata column is wrong. Every row gets the file's `first_row_id` instead of its own value. This happens because `VectorizedSparkOrcReaders.StructConverter` checks `idToConstant` before it checks for metadata columns, so `_row_id` is treated as a plain constant. Values stored in the file are ignored, and for rows with no stored value the row position is not added. `_last_updated_sequence_number` has the same problem. Example: a file with `first_row_id` = 100, where row 0 stores `_row_id` 555 and row 1 stores null. The vectorized reader returns 100 for both. It should return 555 and 101. The row-based ORC reader does this correctly (`OrcValueReaders`, added in #15776 and #16534). The Parquet vectorized reader is also correct. Found while adding vectorized coverage to the format model TCK in #17610. #17635 has an earlier attempt at a fix. ### Willingness to contribute - [ ] I can contribute a fix for this bug independently - [x] I would be willing to contribute a fix for this bug with guidance from the Iceberg community - [ ] I cannot contribute a fix for this bug at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
