RussellSpitzer opened a new pull request #1638: URL: https://github.com/apache/iceberg/pull/1638
The behvaior of hasNonNull in parquet statistics chagnes based on the version of the file that the statistics are read from. For binary columns generated by CDH 1.5.0 to 1.7.0, the function returns hasNonNull as false even if there are nonNull values in the column. This not an issue for files written by OSS Parquet because of a seperate bug which prohibts reading these statistics from Parquet earlier than 1.8.0. The fix for that bug (PARQUET-251) was backported in to the CDH distribution so it correctly can return statistics but not in a format that Iceberg is able to handle correctly. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
