Hi everyone, Thanks again for all the valuable feedback and discussion on this proposal so far.
As we are fast approaching the feature freeze at the end of March, I’ve been thinking about how to best move forward to ensure we can land the core value of this feature in the upcoming release. My proposal is to separate the START_MODE and STATE_RETENTION features entirely. Specifically, I suggest we scope FLIP-557 down to focus solely on START_MODE and move STATE_RETENTION into a completely separate FLIP. It seems we are generally aligned on START_MODE and there isn't much left to debate there. STATE_RETENTION, however, naturally carries more complexity and edge cases. By splitting them into two FLIPs, we allow for the thorough, dedicated discussion that state management requires, without unnecessarily blocking START_MODE from making the cut for this freeze cycle. Let me know what you all think about this approach. If there is general agreement, I can go ahead and update the wiki to reflect the rescoped FLIP-557 and initiate a separate document for state retention. Best regards, Ramin On Mon, Dec 1, 2025 at 6:23 PM Ramin Gharib <[email protected]> wrote: > Hi everyone, > > I would like to start a discussion on FLIP-557: Granular Control over Data > Reprocessing and State Retention in Materialized Table Evolution [1]. > > Currently, ALTER MATERIALIZED TABLE forces a full job restart and > discards state, which is inefficient for many evolution scenarios. FLIP-557 > proposes decoupling data scope from state management by introducing two new > optional clauses: > 1. START_MODE*:* Controls the data processing window (e.g., FROM_BEGINNING, > RESUME_OR_...). > > 2. STATE_RETENTION*:* Controls how existing state is handled (e.g., NONE, > PTF_ONLY). > > This gives users explicit control over cost and correctness during table > evolution. > > For more details, please refer to the FLIP [1]. > > Looking forward to your feedback and thoughts! > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-557%3A+Granular+Control+over+Data+Reprocessing+and+State+Retention+in+Materialized+Table+Evolution > > Best regards, > > Ramin Gharib >
