wgtmac commented on PR #3393: URL: https://github.com/apache/parquet-java/pull/3393#issuecomment-4789434842
I updated the patch so the pruning filters and record-level filters have separate semantics: - Dictionary and Bloom filters now handle NaN literals conservatively and return “might match”. So we don't need to worry about different NaN semantics and boxed NaN values in the dictionary. - Record-level filtering does not special-case NaN. It uses the column’s PrimitiveComparator, so the behavior follows the declared column order: TYPE_DEFINED_ORDER treats NaNs according to the existing type-defined comparator semantics, while IEEE_754_TOTAL_ORDER compares the raw floating-point bits. The same applies to +0/-0. I still think that preserving raw bits of NaN values is a bug fix and benign breaking change. Let me know what you think. @gszadovszky -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
