zhuxiangyi commented on code in PR #9423:
URL: https://github.com/apache/paimon/pull/9423#discussion_r4056639401
##########
paimon-common/src/main/java/org/apache/paimon/predicate/PredicateBuilder.java:
##########
@@ -271,7 +271,9 @@ public Predicate in(int idx, List<Object> literals) {
public Predicate in(Transform transform, List<Object> literals) {
// In the IN predicate, 20 literals are critical for performance.
// If there are more than 20 literals, the performance will decrease.
- if (literals.size() > 20) {
+ // An empty list has no equals to OR together, so it must also take
this branch - mirroring
+ // in(int, List) - rather than fall into or(emptyList()), which throws.
+ if (literals.size() > 20 || literals.isEmpty()) {
Review Comment:
Confirmed, reproduced both forms. `visitIn`/`visitNotIn` now decline the
pushdown outright when `literals` is empty, instead of handing parquet-mr's
`FilterApi` an empty set it refuses. Residual evaluation upstream still applies
the correct always-false/always-true semantics; this layer just stops trying to
prune on it.
Tests:
-
`ParquetFormatReadWriteTest.testNestedInWithEmptyLiteralsDoesNotCrashTheReader`
-
`ParquetFormatReadWriteTest.testNestedNotInWithEmptyLiteralsDoesNotCrashTheReader`
- `ParquetFiltersTest.testNestedInAndNotInWithEmptyLiteralsAreNotPushedDown`
(both forms, filter-level)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
