Tanel Kiis created SPARK-33851:
----------------------------------
Summary: Push partial aggregates bellow exchanges
Key: SPARK-33851
URL: https://issues.apache.org/jira/browse/SPARK-33851
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.2.0
Reporter: Tanel Kiis
The `EnsureRequirements` rule inserts exchanges between partial and final
aggregates to reduce the exchanged data amount. However, if the user manually
partitions the data correctly, then partial aggregates will be after the
exchange.
If possible we should push partial aggregates bellow the manually inserted
exchanges to reduce the exchanged data amount.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
