cetra3 opened a new pull request, #27319: URL: https://github.com/apache/datafusion/pull/27319
## Which issue does this PR close? - Closes #27318. ## Rationale for this change A query that groups by a struct field, and filters with `... OR field IN (subquery)`, fails to plan since #26747: ```text SanityCheckPlan: order and distribution requirements stopped holding after physical optimizer rule 'ProjectionPushdown' ``` #26747 records `get_field(s, k) ≡ __datafusion_extracted_4` when it projects equivalences. `EnsureRequirements` uses that to reuse the join's hash partitioning for the group key, and the plan is valid. Then `ProjectionPushdown` gives the join a projection that drops `__datafusion_extracted_4` but keeps `s`. The join cannot project its partitioning through that projection, because the only way to project a dropped column was through a mapping source in its class. So the partitioning becomes an unknown column, and the final check rejects the plan. ## What changes are included in this PR? - `EquivalenceGroup::project_expr` and `project_expressions` now fall back to another member of the expression's equivalence class when the expression itself cannot be projected. In the issue's plan, the join's partitioning on `__datafusion_extracted_4` projects to `get_field(s, k)`, which the projection above maps to the group key. - The fallback skips literal members, because a literal says nothing about how rows are distributed. The members of a class are equal on every row, so a member describes the same partitioning or ordering as the expression it replaces. #26747 already removes computed members on a join side that is padded with NULLs, so the fallback cannot pick a member that stops being equal after an outer join. ## What is the testing strategy for this PR? - A `sqllogictest` case in `equivalence_properties.slt` runs the query from the issue. It checks the rows and the plan, which keeps the single-phase aggregate with no extra repartition. It fails on `main` with the `SanityCheckPlan` error. - A unit test in `class.rs`, `test_project_dropped_column_through_equivalent_expression`, checks that a projection that drops `a` projects it through `a = b + c`. It fails without the change. - No other expected plan in the `sqllogictest` suite changes. ## Are there any user-facing changes? No. A query that failed to plan now plans. --- Claude Code (an AI coding assistant) wrote this change on behalf of cetra3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
