JingsongLi commented on PR #10175:
URL: https://github.com/apache/paimon/pull/10175#issuecomment-5966507247
Reviewed current head `de846e431f2db05bfffe9d8b2cefda951ad3d8e9`. The
read-plan array filtering use case remains valuable, and the FLOAT `0.1`, NaN
and floating-literal signed-zero cases from the previous review are fixed. One
part of that numeric literal/type finding remains.
**[P2] Normalize DOUBLE numeric literals before the signed-zero comparison**
(`paimon-python/pypaimon/common/predicate_builder.py:63-66`,
`predicate.py:518-524`). Only FLOAT literals are normalized. For an
`ARRAY<DOUBLE>` column, the accepted Python integer literal `0` remains an
`int`, so `_elements_equal(-0.0, 0)` falls back to Python `==` instead of the
signed-zero comparator. The same public query therefore returns different rows
depending on whether its numeric literal is spelled `0` or `0.0`.
On actual written, committed and reopened tables with negative-zero-only
rows 3 and 10, all three methods (`array_contains("doubles", 0)`,
`arrays_overlap("doubles", [0])`, `array_contains_all("doubles", [0])`) return
`[3, 4, 10, 11, 12]`, while their `0.0` versions return the correct
positive-zero rows `[4, 11, 12]`. Projecting only ID and combining the filter
with OR preserve the incorrect extra rows. This is a remaining issue in the new
array APIs, rather than a newly introduced regression in this revision.
For the Java reference, I explicitly normalized `Integer.valueOf(0)` using
`PredicateBuilder.convertJavaObject(DataTypes.DOUBLE(), ...)` and passed the
resulting typed Double to the actual array predicates; those return only `[4,
11, 12]`. Java's direct builder requires typed literals and does not itself
perform that conversion. Please normalize accepted DOUBLE numeric literals too,
or explicitly reject unsupported input types, and cover all three public
methods with persisted-table integer-zero cases.
Validation: 68 Python array/predicate/read/projection tests and 22 Java
PredicateBuilder tests pass; Java tests ran on JDK 8 with normal Maven checks.
Configured flake8 and Python 3.6 grammar checks pass. The additional
actual-file probes reproduce this mismatch in Parquet, Avro, ORC and row
format; current-head CI is green.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
