joyhaldar commented on code in PR #14593:
URL: https://github.com/apache/iceberg/pull/14593#discussion_r2538500140
##########
api/src/main/java/org/apache/iceberg/expressions/InclusiveMetricsEvaluator.java:
##########
@@ -327,6 +327,21 @@ public <T> Boolean eq(Bound<T> term, Literal<T> lit) {
public <T> Boolean notEq(Bound<T> term, Literal<T> lit) {
// because the bounds are not necessarily a min or max value, this
cannot be answered using
// them. notEq(col, X) with (X, Y) doesn't guarantee that X is a value
in col.
+ // However, when min == max and the file has no nulls, we can safely
prune
+ // if that value equals the literal.
+ int id = term.ref().fieldId();
+ if (mayContainNull(id)) {
Review Comment:
Thank you for your review comment @nandorKollar .
You're absolutely right. I've added NaN handling for both cases:
- Files with NaN values in the data
- Files with NaN bounds
Could I request you to please take another look at the PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
