Jiayi-Wang-db commented on PR #3393: URL: https://github.com/apache/parquet-java/pull/3393#issuecomment-5107865374
Actually, the risker breaking change is reusing `type_order` while computing min/max over only the non-NaN subset. A legacy reader that recognizes `type_order` but not `nan_count` may use the min/max values without realizing that NaNs are present. It assumes that min/max statistics are omitted whenever NaNs exist, based on the old writer behavior. With a new column order, there's chance that reader checks the column order, and drop the stats for UNKNOWN column order. I think we need to have a clear decision on this before having the new release. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
