cetra3 opened a new pull request, #27288: URL: https://github.com/apache/datafusion/pull/27288
_Posted by Claude (an AI assistant) on behalf of cetra3._ ## Which issue does this PR close? - Closes #27287. ## Rationale for this change With `datafusion.execution.parquet.pushdown_filters = true`, a query that groups by a `CASE` key and pins it with `HAVING` fails planning with a `SanityCheckPlan` error. See #27287 for a `datafusion-cli` reproduction. The cause is in `FileScanConfig::add_filter_equivalence_info`. It turns every `x = y` conjunct of the source's exact filter into an equivalence class, including `complex_expr = literal`. That puts the literal in a class. `normalize_expr` rewrites bottom-up, so every occurrence of the literal, including the one inside `complex_expr` itself, becomes the canonical expression of the class. In #27287, `b = 'y'` inside the `CASE` becomes `b = <the CASE>`. The partial aggregate requires its input ordered by the group key, because `is_expr_constant` finds the key constant. But the normalized requirement no longer matches the class, so `ordering_satisfy` rejects it. `FilterExec` already avoids this. `collect_columns_from_predicate_inner` keeps only pairs with a column on one side, with a comment that names this exact failure. `FilterExec` then records `complex_expr = literal` as a constant through `ConstExpr::collect_predicate_constants`. The scan path, from #25780, does not have the same guard. Since #26747, ordering checks also consult equivalent expressions, which is where the bisect in #27287 points. ## What changes are included in this PR? `add_filter_equivalence_info` (`datafusion/datasource/src/file_scan_config/mod.rs`) follows `FilterExec`: - An equality with a column on one side becomes an equivalence class, as before. - Any other equality goes through `ConstExpr::collect_predicate_constants`, so `complex_expr = literal` makes `complex_expr` a constant without putting the literal in a class. Only the second kind of conjunct changes. I first added constants for every conjunct, as `FilterExec` does. That changed one expected plan in `push_down_filter_regression.slt` (`group_clustering_mode=Partial([1])` disappeared), so this PR leaves column equalities as they were. ## What is the testing strategy for this PR? - `push_down_filter_regression.slt`: the query from #27287, with a result assertion. It fails with the `SanityCheckPlan` error without this change. - The full sqllogictest suite and the `datafusion-datasource` unit tests pass. ## Are there any user-facing changes? The query shape in #27287 plans and runs. No public API changes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
