Hi all, I would like to restart the discussion thread for FLIP-497: Early Fire Support for Flink SQL Interval Join.
As some of you may recall, this FLIP was originally proposed and went to a vote last year. Because the thread has been open for over a year, I am following the community's recommendation to kick off a fresh discussion (Previous voting thread: Link <https://lists.apache.org/thread/mw0tqgg1yx0mn8tf6msrgdtqbpdpgtpt>). This will ensure the design is fully aligned with the current state of the project and give everyone a chance to review it with fresh eyes. The proposal itself is unchanged from the previous round, where it received positive feedback and no outstanding objections (Link <https://lists.apache.org/thread/dcn78blz7cqmmyypw897w0dvf6l58cnt>). *Motivation (Recap)*: In many streaming applications, particularly real-time analytics and monitoring systems, it is valuable to obtain partial results earlier rather than waiting for full join conditions to finalize. For Flink SQL interval joins, results are typically delayed until watermarks ensure no more matches can occur. This delay can be challenging for scenarios that require fast feedback. Early fire support addresses this by emitting intermediate results speculatively and using retractions or updates to maintain eventual consistency and ensure correctness. This approach can, however, change an append-only stream into a potentially update stream, so the planner and downstream components must be able to handle retraction/upsert messages. Proposal document (aligned with latest master branch): Link <https://docs.google.com/document/d/1YobpNdnvzSsceniVj4NZWi445gb1-54Rox-D7nPArZo/edit?tab=t.0#heading=h.yp9ng89zwc1z> Previous discussion thread: Link <https://lists.apache.org/thread/p3w90rprdtv3vyjog3vl0rql5fvm703j> Current implementation status: Following recent feedback from the community, I have converted the existing implementation PR (Link <https://github.com/apache/flink/pull/28353>) to a draft. I welcome your renewed feedback, architectural insights, and any questions to help refine this feature before we move toward a new voting phase. Thanks, Weiqing
