nyingping created SPARK-38214: --------------------------------- Summary: No need to filter data when the sliding window length is not redundant Key: SPARK-38214 URL: https://issues.apache.org/jira/browse/SPARK-38214 Project: Spark Issue Type: Improvement Components: Structured Streaming Affects Versions: 3.2.1 Reporter: nyingping
At present, the sliding window adopts the form of expand + filter, but in some cases, filter is not necessary. Filter is required if the sliding window is irregular. When the window length is divided by the slide length the result is an integer (I believe this is also the case for most work scenarios in practice for sliding window), there is no need to filter, which can save calculation resources and improve performance. -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org