viirya commented on a change in pull request #31570:
URL: https://github.com/apache/spark/pull/31570#discussion_r576596507
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/AggUtils.scala
##########
@@ -262,6 +262,17 @@ object AggUtils {
* - PartialMerge (now there is at most 1 tuple per group)
* - StateStoreSave (saves the tuple for the next batch)
* - Complete (output the current result of the aggregation)
+ *
+ * Plans a streaming aggregation with Session Window using the following
progression:
+ * - (Shuffle + Session Window Assignment, see `SessionWindowExec`)
+ * - Partial Aggregation (now there is at most 1 tuple per group)
+ * - SessionStateStoreRestore (now there is 1 tuple from this batch +
optionally one from
Review comment:
This process was simplified, compared with original PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
