zuston opened a new pull request, #29952:
URL: https://github.com/apache/flink/pull/29952

   ## What is the purpose of the change
   
   Streaming SQL rejects a query that computes aggregates over several OVER 
windows with different ranges, which is a common pattern in real-time ML 
features engineering:
   
   ```
   SELECT 
     entity_id, 
     event_time, 
     SUM(v) OVER w10, 
     SUM(v) OVER w20, 
     SUM(v) OVER w30 
   FROM events WINDOW 
     w10 AS (PARTITION BY entity_id ORDER BY event_time RANGE BETWEEN INTERVAL 
'10' MINUTE PRECEDING AND CURRENT ROW), 
     w20 AS (PARTITION BY entity_id ORDER BY event_time RANGE BETWEEN INTERVAL 
'20' MINUTE PRECEDING AND CURRENT ROW), 
     w30 AS (PARTITION BY entity_id ORDER BY event_time RANGE BETWEEN INTERVAL 
'30' MINUTE PRECEDING AND CURRENT ROW);
   ```
   
   It fails with "Over Agg: Unsupported use of OVER windows. All aggregates 
must be computed on the same window." Batch mode supports it.
    
   The workaround is one nested subquery per window. Each level is a separate 
operator that stores its own copy of the input rows, so state grows with the 
sum of all ranges (10 + 20 + 30 = 60 minutes of rows instead of 30).
    
   When all windows of one OVER node are bounded event-time RANGE windows with 
the same PARTITION BY and ORDER BY, evaluate them in one operator that:
   1. stores the input rows once, kept for the largest range
   2. keeps one accumulator per window and retracts a row from a window when 
the row leaves that window's range.
    
   Row state becomes max(range_i) instead of sum(range_i). Single-window 
queries keep their current operator and state layout
   
   ## Brief change log
   
   1. Introduce the `RowTimeMultiRangeBoundedPrecedingFunction` to make state 
sharing possible
   
   ## Verifying this change
   
   Test cases cover this change
   
   ## Does this pull request potentially affect one of the following parts:
   
   this will not break down anything and is an internal change.
   
   ## Documentation
   
     - Does this pull request introduce a new feature? (yes / no)
     - If yes, how is the feature documented? (not applicable / docs / JavaDocs 
/ not documented)
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [x] Yes (please specify the tool below)
   
   Generated-by: [Opus 5.5]
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to