adiranjan25 opened a new issue, #18886: URL: https://github.com/apache/iceberg/issues/18886
### Apache Iceberg version None ### Query engine None ### Please describe the bug 🐞 ## Bug description `ParquetDictionaryRowGroupFilter` may incorrectly exclude row groups when evaluating predicates involving positive and negative floating-point zero. Apache Iceberg's expression semantics treat `-0.0` and `+0.0` as equivalent for float and double predicate comparisons. However, the dictionary-filter implementation uses the literal comparator for equality, inequality, and dictionary membership checks. Because Java's floating-point comparison ordering distinguishes the two signed-zero representations, this can produce incorrect filtering decisions. ### Example scenario Consider a Parquet row group where a dictionary-encoded float or double column contains `-0.0`. A query predicate such as: ```sql x = 0.0 ``` should match rows containing `-0.0`. However, when the dictionary filter evaluates the predicate using the default comparator, it may conclude that the dictionary does not contain a matching value and incorrectly skip the row group. ### Expected behavior - `-0.0` and `+0.0` should compare as equivalent for Iceberg float and double predicates. - Dictionary filtering must not incorrectly prune row groups containing matching values. - Equality, inequality, range comparisons, and set predicates should follow Iceberg's expression semantics. ### Proposed investigation and fix 1. Inspect signed-zero comparisons in `ParquetDictionaryRowGroupFilter`, particularly equality, inequality, and range predicates. 2. Reuse the signed-zero-aware comparison semantics introduced in PR #18813 where appropriate, without adding an unnecessary cross-module dependency. 3. Add regression tests using dictionary-encoded float and double values, covering both signed-zero directions. 4. Verify that predicate evaluation never incorrectly excludes a row group containing a matching value. ### Related references - Issue #18723 — Signed-zero predicate-evaluation bug. - PR #18813 — API predicate-evaluation fix. - Discussion: https://github.com/apache/iceberg/pull/18813#issuecomment-6711400628 CC @Rishavbhattarai — thank you for identifying this additional case and offering to help test the follow-up. This issue tracks the Parquet dictionary-filter investigation separately so that PR #18813 can remain focused on API predicate-evaluation correctness. ### Willingness to contribute - [ ] I can contribute a fix for this bug independently - [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community - [ ] I cannot contribute a fix for this bug at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
