andygrove opened a new pull request, #4750: URL: https://github.com/apache/datafusion-comet/pull/4750
## Which issue does this PR close? <!-- No dedicated issue; minor diagnostic improvement. --> N/A ## Rationale for this change When a Spark Final `HashAggregate` cannot be converted to Comet because the partial aggregate ran natively (or would have) and the intermediate buffer formats are incompatible between Spark and Comet, the fallback reason shown in `EXPLAIN` output and the tpcds plan-stability golden files was: ``` HashAggregate [COMET: Spark Final aggregate without Comet Partial requires compatible intermediate buffer formats] ``` This did not tell the reader which aggregate function caused the incompatibility, making it harder to understand why an aggregate fell back to Spark. ## What changes are included in this PR? - Added `QueryPlanSerde.aggsNotSupportingMixedExecution(aggExprs)`, which returns the aggregate functions whose `supportsMixedPartialFinal` is `false` (or that have no Comet serde). `allAggsSupportMixedExecution` is now implemented on top of it so there is a single source of truth. - `CometBaseAggregate.doConvert` now appends the offending function names (deduped `prettyName`s) to the fallback reason, for example: ``` HashAggregate [COMET: Spark Final aggregate without Comet Partial requires compatible intermediate buffer formats, but the following aggregate function(s) have incompatible buffers: count, avg, sum] ``` - Regenerated the affected tpcds plan-stability golden files across all Spark versions. ## How are these changes tested? Covered by the existing `CometTPCDSV1_4_PlanStabilitySuite` / `CometTPCDSV2_7_PlanStabilitySuite` plan-stability tests, whose golden files embed this fallback reason and were regenerated and verified to pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
