[ https://issues.apache.org/jira/browse/SPARK-25555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun updated SPARK-25555: ---------------------------------- Affects Version/s: (was: 2.4.0) 3.0.0 > Generic constructs for windowing and support for custom windows > --------------------------------------------------------------- > > Key: SPARK-25555 > URL: https://issues.apache.org/jira/browse/SPARK-25555 > Project: Spark > Issue Type: Improvement > Components: Structured Streaming > Affects Versions: 3.0.0 > Reporter: Arun Mahadevan > Priority: Major > > Refactor windows logic with generic "assignWindows" and "mergeWindows" > constructs. The existing windows (Tumbling and sliding) and generic session > windows can be built on top of this. It could be extended to support > different types of custom windowing. > K,Values -> AssignWindows (produces [k, v, timestamp, window]) -> GroupByKey > (shuffle) -> MergeWindows (optional step) -> GroupWindows -> aggregate values. -- This message was sent by Atlassian JIRA (v7.6.14#76016) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org