zhangyue19921010 commented on code in PR #11004:
URL: https://github.com/apache/paimon/pull/11004#discussion_r4545850780
##########
paimon-common/src/main/java/org/apache/paimon/predicate/LeafPredicateExtractor.java:
##########
@@ -46,4 +49,25 @@ public Map<String, LeafPredicate> visit(CompoundPredicate
predicate) {
}
return Collections.emptyMap();
}
+
+ /**
+ * Returns the conjuncts that contain a transform leaf, which this
extractor skips and callers
+ * must evaluate on rows, or null if there are none.
+ */
+ @Nullable
+ public static Predicate transformConjuncts(@Nullable Predicate predicate) {
+ List<Predicate> conjuncts =
+ PredicateBuilder.splitAnd(predicate).stream()
+ .filter(LeafPredicateExtractor::containsTransform)
Review Comment:
[P2] Preserve unconsumed sibling conjuncts with the transform residual
On `07a3f3d`, a real `$snapshots` SDK read with `LOWER(commit_kind) =
'append' AND schema_id = 999`, including `newRead().executeFilter()`, returns
both append snapshots with schema ID 0 instead of no rows. The reader only
pushes `snapshot_id`; this filter drops the plain `schema_id` conjunct and
evaluates only `LOWER(commit_kind)`.
The compound query previously failed on its transform, so accepting it while
silently losing a sibling changes the complete-filter behavior. Could these
readers retain all conjuncts their ID optimization does not actually consume,
while preserving the existing Java String IN path? The same structure exists in
the consumers, schemas and manifests readers. I added
`SnapshotsTableTest#testTransformWithUnconsumedConjunctExecuteFilter`: 71
existing tests passed and this additional regression failed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]