LuciferYang opened a new issue, #9646: URL: https://github.com/apache/paimon/issues/9646
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version master, `475be566f` (2.1-SNAPSHOT). ### Compute Engine Spark, through a procedure's `where` argument. ### Minimal reproduce step Pass a procedure filter that compares a column to NULL on both sides of a range: ```sql CALL sys.compact(table => 'db.t', where => 'dt >= 1 AND dt <= null'); ``` The call fails with a `NullPointerException` out of `Between.optimize`. `PredicateBuilder.and` asks it to merge the `>=` and `<=` on the same field, and it compares the two bounds directly: ```java Object lowerBound = greaterOrEqual.literals().get(0); Object upperBound = lessOrEqual.literals().get(0); if (compareLiteral(type, lowerBound, upperBound) >= 0) { ``` `compareLiteral` then runs `((Comparable<Object>) v1).compareTo(v2)`, which throws when `v2` is null; with the null on the other side it falls through to `RuntimeException("Unsupported type")` instead. The procedure path is what makes this reachable: `ExpressionHelper.resolveFilter` runs `ConstantFolding` only, not `NullPropagation` or `ReplaceNullWithFalseInPredicate`, so `dt <= null` survives resolution, and `SparkV2FilterConverter` passes the null literal through. A plain `SELECT ... WHERE dt <= null` does not reach it, because Spark's optimizer folds the comparison to false first. ### What doesn't meet your expectations? A null literal is a legal input elsewhere in the predicate framework, so it should not crash the optimizer. `LeafBinaryFunction.test` already treats a null literal as no match: ```java public boolean test(DataType type, Object field, List<Object> literals) { Object literal = literals.get(0); return field != null && literal != null && test(type, field, literal); } ``` Merging two range predicates into a `BETWEEN` is an optimization; when one bound is null there is nothing to order, and the pair can simply stay unmerged and be evaluated by the rules above. ### Anything else? `compareLiteral` is also called from `NotIn`, `NotBetween` and `LessThan`. Those all null-check the literal before calling, or are only reached with non-null bounds, so `Between.optimize` is the one caller that got there with a null. ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
