yyanyy commented on a change in pull request #1747:
URL: https://github.com/apache/iceberg/pull/1747#discussion_r521838201
##########
File path:
api/src/main/java/org/apache/iceberg/expressions/ManifestEvaluator.java
##########
@@ -142,6 +142,25 @@ public Boolean or(Boolean leftResult, Boolean rightResult)
{
return ROWS_MIGHT_MATCH;
}
+ @Override
+ public <T> Boolean isNaN(BoundReference<T> ref) {
+ int pos = Accessors.toPosition(ref.accessor());
+ // containsNull encodes whether at least one partition value is null,
lowerBound is null if
+ // all partition values are null.
+ ByteBuffer lowerBound = stats.get(pos).lowerBound();
+ if (lowerBound == null) {
Review comment:
You mean check for both `containsNull` and `stats.get(pos).lowerBound()
== null` are true? When would `lowerBound` be null while the column doesn't
contain null? I guess I'll also need to update `notNull` for this too (since I
copied the logic from there)?
##########
File path:
api/src/main/java/org/apache/iceberg/expressions/StrictMetricsEvaluator.java
##########
@@ -144,6 +146,51 @@ public Boolean or(Boolean leftResult, Boolean rightResult)
{
return ROWS_MIGHT_NOT_MATCH;
}
+ @Override
+ @SuppressWarnings("checkstyle:CyclomaticComplexity")
+ public <T> Boolean isNaN(BoundReference<T> ref) {
+ int id = ref.fieldId();
+
+ if (nanCounts != null && nanCounts.containsKey(id) &&
+ valueCounts != null && valueCounts.containsKey(id)) {
+ if (nanCounts.get(id).equals(valueCounts.get(id))) {
+ return ROWS_MUST_MATCH;
+ }
+
+ return ROWS_MIGHT_NOT_MATCH;
+ }
+
+ // for v1 table, when NaN could still be upper/lower bound,
+ // if upper == lower == NaN and null count == 0, the column will only
contain NaN
Review comment:
👍 will remove this block
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
