github-actions[bot] commented on code in PR #68934:
URL: https://github.com/apache/doris/pull/68934#discussion_r4255653607
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergPredicateConverter.java:
##########
@@ -475,7 +507,10 @@ private Types.NestedField getPushdownField(String colName)
{
||
ICEBERG_LAST_UPDATED_SEQUENCE_NUMBER_COL.equalsIgnoreCase(colName)) {
return null;
}
- return schema.caseInsensitiveFindField(colName);
+ Types.NestedField field = schema.caseInsensitiveFindField(colName);
+ // Iceberg 1.11 widens retained four-byte DATE bounds to unscaled
longs. Pruning those
+ // epoch-day bounds with nanosecond predicates can discard matching
files in every planner.
+ return field != null && unsafeNanosFields.contains(field.fieldId()) ?
null : field;
Review Comment:
[P2] Keep unsafe nanos suppression conservative beneath NOT
With `disable_nereids_expression_rules='SIMPLIFY_NOT_EXPR'`, a supported
query retains this plan:
```text
Filter[NOT(ts='2024-01-02' AND id>0)]
IcebergScan(ts promoted from DATE, id)
```
For a v3 unpartitioned table retaining a DATE file with `ts='2024-01-03'`
and `id=1`, this row should survive. The new unsafe-field suppression drops
`ts` in `getPushdownField`; SCAN `buildAnd` keeps `id>0`, then `buildNot`
produces `NOT(id>0)`. Iceberg rewrites that to `id<=0`, and its [metrics
evaluator](https://github.com/apache/iceberg/blob/apache-iceberg-1.11.0/api/src/main/java/org/apache/iceberg/expressions/InclusiveMetricsEvaluator.java#L229-L249)
rejects the file with lower bound `id=1`. The residual cannot recover the
missing row. Drop the whole negated subtree when any child conversion is
inexact, or track conversion exactness through NOT, and cover SDK/cached
planning with this rule disabled. This differs from the existing DATE-bound
thread: it prunes on `id` after suppressing the unsafe timestamp comparison.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
