Jungtaek Lim created SPARK-34889:
------------------------------------
Summary: Introduce MergingSessionsIterator merging elements
directly which belong to the same session
Key: SPARK-34889
URL: https://issues.apache.org/jira/browse/SPARK-34889
Project: Spark
Issue Type: Sub-task
Components: Structured Streaming
Affects Versions: 3.2.0
Reporter: Jungtaek Lim
This issue tracks effort on introducing MergingSessionsIterator, which enables
to merge elements belong to the same session directly. This would be quite
performant compared to UpdatingSessionIterator. Note that
MergingSessionsIterator can only apply to the cases aggregation can be applied
altogether, so there're still rooms for UpdatingSessionIterator to be used.
This issue also introduces MergingSessionsExec which is the physical node on
leveraging MergingSessionsIterator to sort the input rows and aggregate rows
according to the session windows.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
