alamb commented on issue #15002: URL: https://github.com/apache/datafusion/issues/15002#issuecomment-2698440162
the partial aggregation step is used to reduce the data that needs to be passed between cores So even if it is sorted by the group keys, the partial aggregation reduces the amount of data that must be shuffled between cores (aka by the Repartition operator) There is some ascii art here; https://docs.rs/datafusion/latest/datafusion/logical_expr/trait.Accumulator.html#tymethod.state -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
