viirya commented on code in PR #3835:
URL: https://github.com/apache/datafusion-comet/pull/3835#discussion_r3007380057
##########
spark/src/main/scala/org/apache/spark/sql/comet/operators.scala:
##########
@@ -1362,12 +1362,6 @@ trait CometBaseAggregate {
return None
}
- // Aggregate expressions with filter are not supported yet.
- if (aggregateExpressions.exists(_.filter.isDefined)) {
- withInfo(aggregate, "Aggregate expression with filter is not supported")
- return None
- }
Review Comment:
We update native/core/src/execution/planner.rs to construct aggregate filter
expressions and pass to datafusion::physical_plan::aggregates::AggregateExec.
AggregateExec will use these filter expressions in runtime to produce the
filtering boolean array (i.e., the parameter
`Option<&arrow::array::BooleanArray>`) that will be passed to `update_batch`
call.
So DataFusion has this logic internally in AggregateExec.
We only need to make sure the aggregate expressions implemented by Comet
(SumDecimal、SumInt、Avg、AvgDecimal), should apply opt_filter in their
update_batch functions. opt_filter was ignored previously because Comet knows
it won't pass aggregate filter expressions to AggregateExec.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
