Hi all,

I would like to start a discussion on *FLIP-XXX: Support Window Stagger in
FlinkSQL and introduce KEY_BASED deterministic stagger*.

Currently, Flink provides WindowStagger to distribute window trigger time
and reduce burst load (see FLINK-12855: Stagger
TumblingProcessingTimeWindow processing to distribute workload
<https://issues.apache.org/jira/browse/FLINK-12855>). However, there are
two limitations in production usage:

   1. WindowStagger is not supported in Flink SQL TVF tumbling windows
   2. Existing stagger strategies (e.g., RANDOM, NATURAL) are
   non-deterministic and may assign records to different windows after
   recovery, leading to inconsistent results

This FLIP proposes:

   1. Add WindowStagger *support in Flink SQL TUMBLE TVF*
   2. Introduce a new deterministic stagger strategy: *KEY_BASED*
      - The stagger offset is computed from the key
      - Ensures consistent window assignment before and after recovery

Example syntax:

TUMBLE(TABLE t, DESCRIPTOR(rowtime), INTERVAL '1' HOUR, 'RANDOM')

TUMBLE(TABLE t, DESCRIPTOR(rowtime), INTERVAL '1' HOUR, INTERVAL '5'
MINUTE, 'RANDOM')

The detailed designs are described in the FLIP document:

https://docs.google.com/document/d/12NX_s-2HI1C9qjw4two0OHeYfX8YBMiBsq1X6lCfiW0/edit?tab=t.0#heading=h.alqmv4gpvbz7

Looking forward to your feedback.


Best regards,

Zihao Chen

Reply via email to