viirya opened a new pull request, #26819: URL: https://github.com/apache/datafusion/pull/26819
## Which issue does this PR close? - Closes #26818. ## Rationale for this change `AggregateExec` declares `Unique(group keys)` for every aggregation mode, including `Partial` and `PartialReduce`. Partial state is not unique on the group keys. Operators that merge partitions keep the constraint, and a partial stage can also emit the same group more than once within a partition. `ordering_satisfy` treats `[k]` plus `Unique(k)` as satisfying `[k, x]`, so a required sort can be dropped. For example, with `prefer_existing_sort`, a sort on `[k, sum]` above a hash repartition of a Partial aggregate is replaced by an order-preserving repartition, and rows with equal `k` come out unsorted. ## What changes are included in this PR? `AggregateExec::compute_properties` adds the `Unique` constraint only when the mode's output is final (`Final`, `FinalPartitioned`, `Single`, `SinglePartitioned`). Constraints projected from the input are unchanged. Default SQL plans always put the Final aggregate above the Partial one. The Final aggregate keeps declaring `Unique`, and its input ordering requirement covers only the group keys, so this doesn't change any plan in the sqllogictests. ## What is the testing strategy for this PR? - `partial_aggregate_does_not_declare_unique_group_keys` in `physical-plan/src/aggregates/mod.rs`: a Partial aggregate over two sorted partitions declares no constraint, `SortExec([k, sum])` over `SortPreservingMergeExec([k])` sorts the merged rows, and the Final aggregate still declares `Unique`. - `test_keep_sort_above_repartitioned_partial_aggregate` in `core/tests/physical_optimizer/enforce_sorting.rs`: with `prefer_existing_sort`, `EnsureRequirements` keeps the `SortExec` above the hash repartition of the Partial aggregate, and every output partition is sorted. Both tests fail without the change. ## Are there any user-facing changes? No. Partial aggregate output no longer advertises a `Unique` constraint on its group keys. This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
