xndai opened a new issue, #17558: URL: https://github.com/apache/iceberg/issues/17558
### Apache Iceberg version 1.11.0 (latest release) ### Query engine None ### Please describe the bug 🐞 Parquet's Statistics#getNumNulls returns -1 when null_count is missing from the stats. But our current metrics logic doesn't handle this -1 value specifically, instead it just adds to the existing count. For a single row group the -1 total was dropped because Metrics only keeps non-negative counts. But say we have row group 0 that has one null value and row group 1 doesn't have null_count stats and return -1, we end up counting total null count as 0, which is wrong. A query engine that relays on total null count being 0, can skip the file completely when evaluating predicate like WHERE c IS NULL causing wrong results. ### Willingness to contribute - [x] I can contribute a fix for this bug independently - [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community - [ ] I cannot contribute a fix for this bug at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
