JingsongLi opened a new pull request, #9132: URL: https://github.com/apache/paimon/pull/9132
## What changed - remove null literals before building Arrow `IN` expressions and reject null input rows - make `NOT IN` with a null literal match no rows, following SQL three-valued logic - align row-value, file-stats, and Arrow predicate evaluation - add table-read and evaluator-level regression tests for null literals ## Why Arrow set lookup treats null as a matchable set member by default, so `val.isin([None])` does not have SQL `IN (NULL)` semantics. This caused Python reads to return null rows for `IN` predicates and non-matching rows for `NOT IN` predicates containing null. ## Impact Python predicate evaluation now matches Java Paimon and the existing JSON predicate evaluator for `IN` and `NOT IN` with null literals. ## Verification - focused null-literal regression tests - predicate, reader, JSON predicate, projection, data-evolution stats, and pushdown correctness suites: 173 passed, 1 unrelated local pandas 3 dtype assertion deselected, 32 subtests passed - Flake8 on the changed source and test files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
