Github user arunmahadevan commented on the issue:
https://github.com/apache/spark/pull/22482
+1 for the idea to provide native session window support.
On the approach, it would be ideal if all windowing aggregations can be
handled via single plan and state store (v/s the separate plan and state store
the patch proposes for session window). Underlying steps are more or less the
same for Fixed, Session and Sliding windows. The sort/merge operations have to
be part of a window merge function rather than the plan itself.
K,Values -> AssignWindows (produces [k, v, timestamp, window]) ->
GroupByKey (shuffle) -> MergeWindows (optional step) -> GroupWindows ->
aggregate values.
Based on how we want to approach it, it could be handled now or as a follow
up item (with major refactoring).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
