zuston opened a new pull request, #29952:
URL: https://github.com/apache/flink/pull/29952
## What is the purpose of the change
Streaming SQL rejects a query that computes aggregates over several OVER
windows with different ranges, which is a common pattern in real-time ML
features engineering:
```
SELECT
entity_id,
event_time,
SUM(v) OVER w10,
SUM(v) OVER w20,
SUM(v) OVER w30
FROM events WINDOW
w10 AS (PARTITION BY entity_id ORDER BY event_time RANGE BETWEEN INTERVAL
'10' MINUTE PRECEDING AND CURRENT ROW),
w20 AS (PARTITION BY entity_id ORDER BY event_time RANGE BETWEEN INTERVAL
'20' MINUTE PRECEDING AND CURRENT ROW),
w30 AS (PARTITION BY entity_id ORDER BY event_time RANGE BETWEEN INTERVAL
'30' MINUTE PRECEDING AND CURRENT ROW);
```
It fails with "Over Agg: Unsupported use of OVER windows. All aggregates
must be computed on the same window." Batch mode supports it.
The workaround is one nested subquery per window. Each level is a separate
operator that stores its own copy of the input rows, so state grows with the
sum of all ranges (10 + 20 + 30 = 60 minutes of rows instead of 30).
When all windows of one OVER node are bounded event-time RANGE windows with
the same PARTITION BY and ORDER BY, evaluate them in one operator that:
1. stores the input rows once, kept for the largest range
2. keeps one accumulator per window and retracts a row from a window when
the row leaves that window's range.
Row state becomes max(range_i) instead of sum(range_i). Single-window
queries keep their current operator and state layout
## Brief change log
1. Introduce the `RowTimeMultiRangeBoundedPrecedingFunction` to make state
sharing possible
## Verifying this change
Test cases cover this change
## Does this pull request potentially affect one of the following parts:
this will not break down anything and is an internal change.
## Documentation
- Does this pull request introduce a new feature? (yes / no)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs
/ not documented)
---
##### Was generative AI tooling used to co-author this PR?
- [x] Yes (please specify the tool below)
Generated-by: [Opus 5.5]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]