JingsongLi commented on code in PR #9423:
URL: https://github.com/apache/paimon/pull/9423#discussion_r4056524230
##########
paimon-common/src/main/java/org/apache/paimon/predicate/PredicateBuilder.java:
##########
@@ -271,7 +271,9 @@ public Predicate in(int idx, List<Object> literals) {
public Predicate in(Transform transform, List<Object> literals) {
// In the IN predicate, 20 literals are critical for performance.
// If there are more than 20 literals, the performance will decrease.
- if (literals.size() > 20) {
+ // An empty list has no equals to OR together, so it must also take
this branch - mirroring
+ // in(int, List) - rather than fall into or(emptyList()), which throws.
+ if (literals.size() > 20 || literals.isEmpty()) {
Review Comment:
[P2] Keep empty Transform sets from reaching parquet-mr
This fixes builder evaluation, but it now lets an empty In/NotIn leaf reach
the Parquet consumer. Both ParquetFilters.visitIn and visitNotIn call FilterApi
with an empty Set; parquet-mr 1.16 rejects that in SetColumnFilterPredicate
with IllegalArgumentException, and ParquetFilters.convert catches only
UnsupportedOperationException. I reproduced both nested IN(empty) and NOT
IN(empty) failing while creating the Parquet reader, so the new overload still
cannot be used end to end with a dynamically empty literal list. Please have
the Parquet visitor treat empty sets as unsupported, preserving residual
evaluation, and add nested Parquet reader regressions for both forms.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
