Hi Weiqing, I've just read through the whole FLIP and +1 on the direction.
I've a comment apart from the other pending items. Namely the configuration is `state.schema-evolution.enable` which implied to me that it's a generic state evolution feature but it's limited to Row data. Maybe we can mark that it's Row data specific. I'm pretty sure that we're going to add further types but not all. BR, G On 2025/04/26 05:45:32 Weiqing Yang wrote: > Hi all, > > I’d like to initiate a discussion about enhancing state schema evolution > support for RowData in Flink. > > *Motivation* > > Flink applications frequently need to evolve their state schema as business > requirements change. Currently, when users update a Table API or SQL job > with schema changes involving RowData types (particularly nested > structures), they encounter serialization compatibility errors during state > restoration, causing job failures.The issue occurs because existing state > migration mechanisms don't properly handle RowData types during schema > evolution, preventing users from making backward-compatible changes like: > > - > > Adding nullable fields to existing structures > - > > Reordering fields within a row while preserving field names > - > > Evolving nested row structures > > This limitation impacts production applications using Flink's Table API, as > the RowData type is central to this interface. Users are forced to choose > between maintaining outdated schemas or reprocessing all state data when > schema changes are required. > > Here’s the proposal document: Link > <https://docs.google.com/document/d/1WtAxp-jAVTLMOfWNldLCAoK137P0ZCMxR8hOZGcMxuc/edit?tab=t.0> > Your feedback and ideas are welcome to refine this feature. > > Thanks, > Weiqing >
