chenjian2664 commented on code in PR #11004:
URL: https://github.com/apache/paimon/pull/11004#discussion_r4491032323
##########
paimon-common/src/main/java/org/apache/paimon/predicate/LeafPredicateExtractor.java:
##########
@@ -30,11 +29,11 @@ public class LeafPredicateExtractor implements
PredicateVisitor<Map<String, Leaf
@Override
public Map<String, LeafPredicate> visit(LeafPredicate predicate) {
- Optional<FieldRef> fieldRefOptional = predicate.fieldRefOptional();
- if (!fieldRefOptional.isPresent()) {
- throw new UnsupportedOperationException();
- }
- return Collections.singletonMap(fieldRefOptional.get().name(),
predicate);
+ // A leaf on a transform such as UPPER(name) does not constrain the
field itself.
+ return predicate
+ .fieldRefOptional()
+ .map(fieldRef -> Collections.singletonMap(fieldRef.name(),
predicate))
Review Comment:
Good catch, thanks. `FilesRead` and `FileKeyRangesRead` excluded every
conjunct naming `partition`, `bucket` or `level`, but the scan only pushes
top-level leaves on the plain field. A transform such as `UPPER(partition)` was
therefore applied nowhere, and the same was true for an `OR` on those fields.
Both now use a shared `FilesTable.scanResidualPredicates`, which drops only the
leaves the scan pushes and evaluates the rest on the rows. I added regressions
for `$files` and `$file_key_ranges` with `UPPER(partition) = ...`.
One case is unchanged. With two plain leaves on the same field, such as a
partition range, the scan keeps only one, yet both are still dropped from the
row filter. The code behaved this way before this PR too. I can follow up
separately if you'd like.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]