jonasHanhan opened a new pull request, #27073: URL: https://github.com/apache/datafusion/pull/27073
## Which issue does this PR close? - Closes #27058. ## Rationale for this change `GROUP BY k, k` is legal SQL and does not change the result, but producing Substrait from that plan and consuming it again swapped the grouping and measure columns. DataFusion's `Aggregate` schema keeps each grouping key once (`grouping_set_to_exprlist`). The producer already shared identical expressions in `grouping_expressions`, but it still wrote a repeated `expression_reference` for each occurrence (`[0, 0]`). The consumer expands each reference into its own output column, so the AggregateRel effectively gained an extra column. Projections built from the DataFusion plan still indexed fields as if that column were not there, and a round trip returned `(sum, k)` under the names `(k, s)`. ## What changes are included in this PR? In `parse_flat_grouping_exprs`, skip grouping expressions that are already present in the current grouping. The AggregateRel then declares and references the repeated key once, matching the DataFusion Aggregate schema that projections are built against. ## What is the testing strategy for this PR? Added `aggregate_duplicate_group_by_key_roundtrip` in `roundtrip_logical_plan.rs`. It uses an unoptimized plan so `eliminate_duplicated_expr` does not hide the bug, checks that the producer emits a single grouping expression reference, and asserts that collecting the plan before and after a Substrait round trip returns the same batches. It also covers the optimized path where the duplicate key is already removed. Locally: `cargo test -p datafusion-substrait` passes, and `cargo clippy -p datafusion-substrait --all-targets -- -D warnings` and `cargo fmt` are clean. ## Are there any user-facing changes? Substrait plans produced from logical plans that still contain repeated `GROUP BY` keys (for example unoptimized plans) now round-trip with the correct column order. No API changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
