tdas commented on a change in pull request #33093:
URL: https://github.com/apache/spark/pull/33093#discussion_r661571317
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FlatMapGroupsWithStateExec.scala
##########
@@ -250,6 +370,63 @@ case class FlatMapGroupsWithStateExec(
}
}
- override protected def withNewChildInternal(newChild: SparkPlan):
FlatMapGroupsWithStateExec =
- copy(child = newChild)
+ override protected def withNewChildrenInternal(
+ newLeft: SparkPlan, newRight: SparkPlan): FlatMapGroupsWithStateExec =
+ copy(child = newLeft, initialState = newRight)
+}
+
+object FlatMapGroupsWithStateExec {
+
+ /**
+ * This method merges two grouped iterators such that we have a combined
iterator
Review comment:
it does not just combine.. it assumed the grouping iterator is sorted by
key (first tuple in the input (InternalRow, Iterator[InterenalRow]) using the
given comparator and then merges them according to that sorted order. you have
to explain this bettr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
