HeartSaVioR commented on a change in pull request #35673:
URL: https://github.com/apache/spark/pull/35673#discussion_r815951144
##########
File path:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingAggregationSuite.scala
##########
@@ -542,8 +543,8 @@ class StreamingAggregationSuite extends
StateStoreMetricsTest with Assertions {
/**
* This method verifies certain properties in the SparkPlan of a streaming
aggregation.
* First of all, it checks that the child of a `StateStoreRestoreExec`
creates the desired
- * data distribution, where the child could be an Exchange, or a
`HashAggregateExec` which already
- * provides the expected data distribution.
+ * data distribution, where the child is a `HashAggregateExec` which already
provides
Review comment:
The change of this test is intended - the main change leads the
streaming aggregation to always introduce shuffle just after partial
aggregation, so that remaining parts execute in the same stage with same output
partitioning. The test was relying on old behavior which triggers shuffle just
before state store load node in some case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
