ranflarion opened a new pull request, #26083: URL: https://github.com/apache/datafusion/pull/26083
## Which issue does this PR close? - Closes #26082. ## Rationale for this change `n IN (2, 100, 101, CASE WHEN d > 5 THEN 1 ELSE d END * 5)` returns wrong results on every row (repro in #26082). `try_evaluate_constant_list` decides that a list is constant by evaluating it on an empty batch, and `CaseExpr` returns its THEN branch as a scalar for a zero-length WHEN mask, so the CASE item is frozen to `5` in the static filter and never evaluated per row. ## What changes are included in this PR? `try_evaluate_constant_list` returns `None` unless every leaf of every list item is a `Literal`, so any other list takes the per-batch evaluation `InListExpr` already uses for items that return arrays. Lists of literals, casts of literals and other expressions over literals still build the static filter from their empty-batch evaluation, as before. Checking leaves rather than `collect_columns` keeps the rule independent of which leaf types read the input (`Column`, `LambdaVariable`, or a leaf defined outside DataFusion). The cost is that an item with a non-literal leaf that is nonetheless constant, such as a stable function the optimizer did not fold, is now evaluated per batch instead of once. ## What is the testing strategy for this PR? A new section in `in_list.slt` runs the repro as a projection and as a filter, and `test_in_list_case_item_reading_a_column` builds the same expression through `in_list()`. Both fail on main and pass with this change. The other `in_list` unit tests, the `physical-expr` lib tests, the full sqllogictest suite and the `filter_pushdown` integration tests pass unchanged. ## Are there any user-facing changes? No API changes. IN lists whose items have a non-literal leaf return correct results, evaluated per batch rather than through the static filter. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
