Fokko commented on PR #3393: URL: https://github.com/apache/parquet-java/pull/3393#issuecomment-5123209946
Joining the party late here. I just checked, and for example 1.12.x would ignore the min/max statistics when it encounters an unknown sort order (according to Claude Opus): > So it's safe and forward-compatible: 1.12.x reads all the data correctly but conservatively declines to use the min/max stats it can't interpret. The only "cost" is lost statistics-based pruning for that column — which is the intended, correct behavior, since 1.12.x has no idea how IEEE 754 total order sorts those values and using them under the wrong ordering assumption could produce incorrect query results. The tricky part is that we don't know how other readers in the wild are handling this, and we want to avoid triggering NPE's (or any other language equivalent). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
