Copilot commented on code in PR #12741:
URL: https://github.com/apache/gluten/pull/12741#discussion_r3749580831
##########
gluten-ut/test/src/test/scala/org/apache/gluten/execution/MergeTwoPhasesHashBaseAggregateSuite.scala:
##########
@@ -113,6 +113,43 @@ abstract class BaseMergeTwoPhasesHashBaseAggregateSuite
extends WholeStageTransf
compareResult = true,
df => checkHashAggregateCount(df, 1)
)
+
+ // pure distinct + FILTER: Spark's planAggregateWithOneDistinct inserts
an exchange
+ // between the partial and final stages, so the merge rule's direct
parent-child pattern
+ // match does NOT fire. This records that boundary: distinct + FILTER is
not merged into
+ // a single aggregate, and the FILTER is preserved through the unmerged
stages.
+ compareResultsAgainstVanillaSpark(
+ """
+ |SELECT count(DISTINCT key) FILTER (WHERE key > 50) AS pc
+ |FROM v1
+ |""".stripMargin,
+ compareResult = true,
+ df => {
+ df.collect()
+ val plans = collect(df.queryExecution.executedPlan) {
+ case agg: HashAggregateExecBaseTransformer => agg
+ }
+ assert(plans.size > 1, "distinct + FILTER should not be merged into
a single aggregate")
+ }
Review Comment:
`compareResultsAgainstVanillaSpark` already triggers an action (`collect`)
before invoking `customCheck`, so the extra `df.collect()` here causes the
query to execute an additional time without affecting the plan inspection.
Removing it will make the test faster and avoid redundant work.
This issue also appears on line 145 of the same file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
