Hi Gyula, +1 to adding field names to a new RowDataSerializerSnapshot version.
Persisting the names in a new snapshot version sounds like the cleanest solution for this use case. +1 also to keeping the operator-level keyed-state table while adding flattened tables for list/map states. BTW, I think we should use the state name as the user-facing part of the per-keyed-state table name. State names are what users specify in StateDescriptor and are usually the most meaningful identifier when debugging state. Since state names are only unique within an operator, the physical table name may still need an operator uid/hash prefix or a collision-handling rule, but the state name should be visible in the table name. For example: - operator table: uid_<operator_uid>_keyed(key, value_state, list_state, map_state) - flattened list state table: uid_<operator_uid>_<state_name>_list(key, index, value) - flattened map state table: uid_<operator_uid>_<state_name>_map(key, map_key, value) Best, Shengkai Roman Khachatryan <[email protected]> 于2026年7月8日周三 03:39写道： > > 1. The catalog built on the regular state processor api (and therefore > > flink state restore) capabilities has limited scope to detect exactly > what > > happens when a state is no longer there. This will probably lead to read > > errors/not found exceptions etc, some of which happens in code that is a > > bit tricky to control this way. Let's see how well this works in practice > > and error handling can always be improved in general. This is not part of > > the catalog design itself. > > Makes sense. I think this can also be an extension. > To clarify the ownership/pinning follow-up idea: it could use FS mechanisms > rather than HA (ZK/etcd). For example, a separate file > with the lowest checkpoint that Flink should keep, protected by CAS and > limited to a specified TTL. > > > 2. The proposal in it's current form includes a global state metadata > view > > based on the existing metadata information ([1]) and based on Shengkai's > > feedback a per operator granular metadata table/view that would expose > > information of individual states. I don't see where file level > information > > fits into this but if you have a good way / idea how to represent this as > a > > table this can definitely be a future extension/addition > > I was thinking about something like: > > USE `00000000000000000000/chk-42`; > SELECT * > FROM state_handles; > > -- id type parent path > size timestamp operator/state/subtask_index > local_path key_range > -- 0 FileStateHandle - s3://.../_metadata > 0.8Kb 2026-07-02T22:56:30 - > - - > -- 1 IncrementalRemoteKeyedStateHandle 0 > - 2026-07-02T22:56:14 > uid_transaction_aggregator_keyed/users#0 - 0 .. 127 > -- 2 FileStateHandle 1 > s3://.../xxxx-xxxx... 5.4Mb 2026-07-02T22:56:12 - > 000034.SST - > -- 3 FileStateHandle 1 > s3://.../yyyy-yyyy... 872Kb 2026-07-02T22:56:12 - > 000035.SST - > -- 4 IncrementalRemoteKeyedStateHandle 0 > - 2026-07-02T22:56:24 > uid_transaction_aggregator_keyed/users#1 - 128 .. 255 > -- 5 ByteStreamStateHandle 4 > 2Kb 2026-07-02T22:56:24 - > 000001.SST - > > The idea is to represent CompletedCheckpoint as a DAG so that it maps > directly to the layout of the object in-memory. > > > 3. Did not think about this but if this becomes a requirement we could > add > > a flag to enable metadata only in the catalog. > > For our use-case (multi-tenant cloud environment), separate access models > for > data and metadata are very likely a must have because of the security > concerns: > - internally, the operators should have access to metadata, but not to the > customer data > - externally, the users should have access to their data but not to the > metadata > > 5, 6. Thanks! :) > > 7. Yes, this should be available since metadata V4. > > Regards, > Roman > > > On Tue, Jul 7, 2026 at 9:56 AM Gyula Fóra <[email protected]> wrote: > > > Hey Shengkai! > > > > Thanks for the questions, you hit on some very good practical points. Let > > me provide my answers below, in the meantime I have already updated the > > FLIP to include some of your suggestions :) > > > > 1. How would schema inference work for RowDataSerializer? > > > > That's a good observation, I did not notice this. Probably the simplest > > solution would be to introduce a new version in the > > RowDataSerializerSnapshot and include the names for this use-case. > > This would not really impact checkpointing times/performance but would > > allow a straightforward mapping for sql states. > > > > If we feel that this is too much internal change, then we could also keep > > it as is for now using simply f0, f1... > > > > 2. Is one keyed-state table per operator the right abstraction? > > > > This is a very good point and something that has bothered me as well > from a > > design perspective. There is no single good abstraction here because > there > > are completely different use cases. > > When you just want to look into the state for a single / multiple keys > and > > you mostly have simple value list states, the single table representation > > is superior from both query syntax and performance perspective. It avoids > > JOINS and maps to the simple mental model that for a certain key you have > > state x,y,z. Due to this straightforward mental model I still think this > is > > the good default representation. With projection pushdown, it's easy to > > select one/several specific columns without reading / touching any other. > > > > The big issue is however with large collection states that simply cannot > be > > represented within a single row. This happens very often and is one of > the > > main reasons someone would even use a list state (if they understand how > > they work internally, but not all users do...). Large map, window, list > > states won't work in the simple row model and are completely impractical. > > > > Based on this, my recommendation would be to keep all keyed states in a > > single table as per the original proposal (one column per keyed state) > but > > also add an extra table per list / map state with the flattened schema. > > So if the operator has a value and list state, then there would be 2 > > tables. One with both states as columns (as per original design) + 1 > > flattened list state table (key, index, value) or for map states (key, > > map_key, value). > > > > This way we cover both use cases naturally. I am also open to making this > > configurable on the catalog level. > > > > I have added this to the FLIP > > > > 3. Could you clarify the assumption of "reading state without user > > classes"? > > > > Turns out from an implementation perspective it's not too bad and > pojo/avro > > state schemas can be inferred quite naturally for most cases. However if > > the user indeed provides the user jar on the classpath then the whole > > schema resolution will become even simpler because then we do not need > any > > custom inference. For our own use-cases and in general I would not like > to > > assume that user classes will be easily available or that a catalog will > > represent mostly a single application. On the contrary the way We intend > to > > use this, is definitely mostly without userjars and to represent multiple > > applications at the same time. > > > > 4. Could StateCatalog expose more fine-grained metadata? > > > > I think this is a very good idea. I have updated the FLIP to include an > > operator level metadata table as well (one for each operator). I would > love > > to include everything that you suggested, I think the practical limit is > > what kind of information is part of the checkpoint and what isn't . This > > also ties to some questions Roman had about more detailed metadata. Makes > > sense > > > > Cheers > > Gyula > > > > > > On Tue, Jul 7, 2026 at 4:59 AM Shengkai Fang <[email protected]> wrote: > > > > > Hi Gyula, > > > > > > Thanks for the FLIP. I like the direction of making > savepoint/checkpoint > > > state discoverable and queryable from SQL. I have a few questions and > > > concerns about the proposed abstraction. > > > > > > 1. How would schema inference work for RowDataSerializer? > > > > > > From the current RowDataSerializer snapshot, it looks like the snapshot > > > persists the LogicalType[] and nested serializer snapshots, so the > field > > > types can be restored. However, the top-level RowDataSerializer > > constructed > > > from a RowType seems to store only the child LogicalTypes, not the > > RowType > > > field names. Would StateCatalog expose generated names such as f0/f1, > or > > is > > > there another source for recovering the original field names? > > > > > > 2. Is one keyed-state table per operator the right abstraction? > > > > > > I wonder whether one table per named keyed state would be a better base > > > abstraction, with an optional operator-level wide view on top. In > > > particular, MapState can contain an unbounded or highly variable number > > of > > > entries per state key. Exposing it as a MAP<K,V> column may require > fully > > > deserializing the map for a key into heap. A normalized table shape > such > > > as: > > > > > > (state_key, map_key, map_value) > > > > > > seems more scalable and SQL-friendly for MapState. Similarly, > > > ValueState/ListState/MapState have different natural table shapes, so > > tying > > > the physical table boundary to the operator may be too coarse. > > > > > > 3. Could you clarify the assumption of "reading state without user > > > classes"? > > > > > > This is a very attractive goal, but it also seems to introduce > > substantial > > > complexity for POJOs, Avro SpecificRecord, subclasses, and custom > > > serializers. If StateCatalog is positioned as a > > job-level/application-level > > > catalog, would requiring the job jar or user artifacts be acceptable > as a > > > first step? That might simplify the design while still covering many > > > operational/debugging use cases. > > > > > > 4. Could StateCatalog expose more fine-grained metadata? > > > > > > For debugging state, it would be useful to expose state-level metadata > > such > > > as state name, state type, serializer snapshot/serializer class, TTL > > > configuration, namespace/window information where applicable, backend > > state > > > type, and possibly whether a state can be read lazily/streamingly. > > > > > > Best, > > > Shengkai > > > > > > Roman Khachatryan <[email protected]> 于2026年7月7日周二 08:44写道： > > > > > > > Hi Gyula, > > > > > > > > Thanks for the proposal, this looks very useful! A few questions and > > > > comments: > > > > > > > > 1. Following up on Han's question about checkpoint retention: I > > > understand > > > > external coordination is out of scope for now, but could the catalog > at > > > > least detect that a checkpoint was subsumed/deleted mid-query and > fail > > > with > > > > a clear error, rather than a low-level file-not-found? And do you see > > > > ownership/pinning as a possible follow-up FLIP once checkpoint > reading > > > > picks up adoption? > > > > 2. Does the proposal allow querying file-level metadata (file size, > > > > creation date, etc.)? This would be useful for debugging > > > compaction-related > > > > issues. > > > > 3. If yes, could data and metadata queries have separate access > modes? > > In > > > > many environments access to data is much stricter than access to > > > metadata, > > > > so being able to grant metadata-only access to the catalog would > > broaden > > > > where it can be deployed. > > > > 4. Just to confirm: incremental checkpoints are expected to work > > through > > > > the regular restore mechanisms, given sufficient retention? > > > > 5. +1 on bringing non-keyed state into scope — a concrete use case: > > > > inspecting Kafka transaction state (currently stored in non-keyed > > > operator > > > > state) would be very valuable for debugging EOS issues. > > > > 6. Could you explain why timers are not supported? They live in keyed > > > state > > > > and the state processor API can read registered timers, so I'm > > wondering > > > > whether this is a fundamental limitation or just table-mapping scope. > > > > 7. Does the proposal allow querying checkpoint metadata (such as > > > > SharingFilesStrategy, isSavepoint, etc.)? This could be useful for > > > > debugging CLAIM mode issues. > > > > > > > > > > > > Regards, > > > > Roman > > > > > > > > > > > > On Mon, Jul 6, 2026 at 1:02 PM Gyula Fóra <[email protected]> > > wrote: > > > > > > > > > Hi Zakelly! > > > > > > > > > > That's a good point and we have to ensure that it works. In theory > > SQL > > > > > related states are relatively easy to cover and represent. The > > RowData > > > > > state would be mapped directly to ROW<...> similar to other pojo > key > > > > > states. > > > > > > > > > > Cheers > > > > > Gyula > > > > > > > > > > On Sun, Jul 5, 2026 at 4:03 PM Zakelly Lan <[email protected]> > > > > wrote: > > > > > > > > > > > Hi Gyula, > > > > > > > > > > > > Thanks for driving this, it's a nice addition and I fully support > > it. > > > > One > > > > > > thing to make sure: > > > > > > > > > > > > For the state generated by some Flink SQL jobs, does the > > StateCatalog > > > > > infer > > > > > > this internal `RowData` structure and expose it as a SQL > `ROW<...>` > > > > type? > > > > > > For example, a regular streaming join side may be stored as a > state > > > > such > > > > > as > > > > > > `left-records` / `right-records`, whose value or map key/value > > > > contains a > > > > > > `RowData` for the original input row. > > > > > > > > > > > > > > > > > > Best, > > > > > > Zakelly > > > > > > > > > > > > On Fri, Jul 3, 2026 at 4:13 PM Dennis-Mircea Ciupitu < > > > > > > [email protected]> wrote: > > > > > > > > > > > > > Hi Gyula, > > > > > > > > > > > > > > Thanks for the detailed answers. This addresses my questions > well > > > and > > > > > the > > > > > > > direction sounds great. > > > > > > > > > > > > > > +1 (non-binding) from my side. > > > > > > > > > > > > > > Best regards, > > > > > > > Dennis > > > > > > > > > > > > > > > > > > > > > On Thu, Jul 2, 2026 at 3:26 PM Gyula Fóra < > [email protected]> > > > > > wrote: > > > > > > > > > > > > > > > Hi Dennis! > > > > > > > > > > > > > > > > Thank you for the questions. Much recent work in the state > > > > connector > > > > > > api > > > > > > > > has been done basically towards this type of nice cataloging > > and > > > > > > flexible > > > > > > > > access. There are a few holes and things that have to be > > changed, > > > > not > > > > > > > > everything is enumerated in the FLIP but we have to have an > > open > > > > mind > > > > > > and > > > > > > > > make all necessary changes as you said to make this truly > nice > > > and > > > > > > > > comprehensive as much as possible. Most state processor apis > > are > > > > > marked > > > > > > > > experimental so we can be flexible within reason :) > > > > > > > > > > > > > > > > Now to the concrete questions: > > > > > > > > > > > > > > > > 1. Non-keyed state support / scope > > > > > > > > I think non-keyed states should definitely be in the scope of > > the > > > > > FLIP > > > > > > in > > > > > > > > terms of design , and my intention was not to exclude them I > > just > > > > > > focused > > > > > > > > on the keyed state as that is readily available in our > > prototype > > > > > > > > implementation (without much changes to the existing > > > connectors). I > > > > > > will > > > > > > > > try to update the FLIP to include non-keyed states more in > > detail > > > > > but I > > > > > > > > think the case is pretty straightforward. From a table > > > > representation > > > > > > > > perspective, they can follow a similar pattern such as: > > > > > > > > uid_opUID_statename_broadcast , uid_opUID_statename_list . A > > > > > > > corresponding > > > > > > > > SQL connector can easily be added to support these based on > the > > > > > > existing > > > > > > > > datastream connector. I will make sure to add separate > tickets > > > for > > > > > > these > > > > > > > > types of states once the FLIP is accepted and this work can > > very > > > > > easily > > > > > > > be > > > > > > > > parallelized across different state types within the existing > > > > catalog > > > > > > > > frameworks. This way keyed/non-keyed states will live > directly > > > > > together > > > > > > > in > > > > > > > > a single catalog/db. > > > > > > > > > > > > > > > > In the future we can even go a step further and include > > connector > > > > > > > specific > > > > > > > > state views such as kafka offsets etc with custom connector > > > > specific > > > > > > > > plugins > > > > > > > > > > > > > > > > 2/3. Serializer transparency and robustness > > > > > > > > From a practical standpoint both generated (synthetic) > > > serializers > > > > > and > > > > > > > > custom classes / kryo and pluggable logic could work but the > > > whole > > > > > > > catalog > > > > > > > > concepts requires a certain behaviour to be useful. The > catalog > > > > would > > > > > > > point > > > > > > > > to savepoint directories and discover all state in it > > > (potentially > > > > > from > > > > > > > > multiple jobs). Configuration has to be done in a generic > way, > > I > > > > > don't > > > > > > > see > > > > > > > > a problem with introducing configs for specifying custom > > > > > > > > serializers/factories either generically for certain specific > > > > > classes. > > > > > > In > > > > > > > > most cases however this won't be necessary as the state > > snapshot > > > > > itself > > > > > > > > usually has a reference (classname) of the original user > > classes. > > > > If > > > > > > the > > > > > > > > catalog process has access to those classes it will use that > > > > > directly, > > > > > > or > > > > > > > > other confugred serializers, and only if not available fall > > back > > > to > > > > > > > > generating serializers for POJO/TUPLE types. There is > > obviously a > > > > > limit > > > > > > > to > > > > > > > > what is possible here initially, Kryo being one exception > where > > > you > > > > > > > either > > > > > > > > have the class or not. > > > > > > > > > > > > > > > > I would like to however point out that we do not have to > > support > > > > > > > everything > > > > > > > > initially, we can start with what is currently available, use > > the > > > > > > > classpath > > > > > > > > / generated serializers and as we develop we will find the > > limits > > > > of > > > > > > this > > > > > > > > approach and then can extend with configuration as it feels > > > natural > > > > > > > instead > > > > > > > > of trying to create a super complex initial solution. But I > > > > > definitely > > > > > > > > agree that we should support custom serializer already > > specified > > > in > > > > > the > > > > > > > > config that is otherwise used by flink for the jobs (but I > > think > > > > this > > > > > > > > should more or less work out of the box). > > > > > > > > > > > > > > > > 4. The metadata view is currently reused based on the > existing > > > > table > > > > > > > valued > > > > > > > > function. Let's take this as a followup under this umbrella > to > > > > > improve > > > > > > / > > > > > > > > extend the metadata view. I don't think we need a separate > FLIP > > > but > > > > > it > > > > > > > also > > > > > > > > feels out of scope here. > > > > > > > > > > > > > > > > Cheers > > > > > > > > Gyula > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jul 2, 2026 at 1:02 PM Dennis-Mircea Ciupitu < > > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > Thank you for driving this. Being able to discover > > > > > > > savepoints/checkpoints > > > > > > > > > and query their state as SQL tables without shipping the > > > original > > > > > > user > > > > > > > > > classes is a genuinely valuable addition, and it's nice > that > > it > > > > > > builds > > > > > > > on > > > > > > > > > the existing state-table connector and savepoint_metadata > > work > > > > > rather > > > > > > > > than > > > > > > > > > starting from scratch. > > > > > > > > > > > > > > > > > > A few points and questions, mostly around scope and the > > > > serializer > > > > > > > story: > > > > > > > > > > > > > > > > > > 1. Non-keyed state and the DataStream path. > > > > > > > > > - The FLIP scopes out BroadcastState, operator > > ListState > > > > and > > > > > > > > > UnionState because "no readily available Table API > > > > connectors > > > > > > > exist > > > > > > > > > for > > > > > > > > > these state types." That's a fair characterization of > > the > > > > > Table > > > > > > > > > layer, but > > > > > > > > > the state-processor DataStream API already reads all > > > three > > > > > > today > > > > > > > > > (SavepointReader#readBroadcastState / > #readUnionState / > > > > > > > > > #readListState). So > > > > > > > > > the limitation is really in the keyed-only SQL > mapping > > > > > > > > > (KeyedStateReader > > > > > > > > > runs inside a keyed backend), not in the snapshots > > > > > themselves. > > > > > > > > > - Is the keyed-only scope a deliberate > UX/table-mapping > > > > > > decision, > > > > > > > > or > > > > > > > > > would a DataStream-backed reader be considered so the > > > > catalog > > > > > > > isn't > > > > > > > > > strictly less capable than the API it extends? Even > if > > > > > > non-keyed > > > > > > > > > contents > > > > > > > > > stay out of scope initially, it would be good to > frame > > > this > > > > > > > > > explicitly as a > > > > > > > > > Table-mapping constraint rather than a general one. > > > > > > > > > 2. Serializer transparency - the "no user classes" > premise > > > vs. > > > > > > > custom > > > > > > > > > serializers. > > > > > > > > > - The design relies on Flink's transparent serializer > > > > formats > > > > > > to > > > > > > > > > decode state without user dependencies, which is > great > > > for > > > > > > > > > POJO/Avro/basic > > > > > > > > > types. But two serialization efforts point the other > > way: > > > > > > > FLIP-398 > > > > > > > > > [1] > > > > > > > > > (released) already lets users configure serializers > per > > > > type > > > > > > via > > > > > > > > > pipeline.serialization-config, and FLIP-538 [2] (in > > > > > discussion) > > > > > > > > adds > > > > > > > > > pluggable custom generic-type serializers (e.g. > Apache > > > > Fory) > > > > > > and > > > > > > > > > promotes > > > > > > > > > TypeSerializer/TypeSerializerSnapshot to @Public. As > > > > FLIP-538 > > > > > > > > > itself notes, > > > > > > > > > state written with a custom serializer becomes > > dependent > > > on > > > > > > that > > > > > > > > > serializer > > > > > > > > > to decode - external tooling without it cannot read > > those > > > > > > bytes. > > > > > > > > > - Could we make the deserialization side pluggable > and > > > > > > > > config-driven, > > > > > > > > > mirroring FLIP-398's serialization-config, with a > > > graceful > > > > > > > fallback > > > > > > > > > (e.g. > > > > > > > > > expose the raw bytes / skip the column) when a format > > > isn't > > > > > > > > > transparently > > > > > > > > > decodable? There already seems to be a seam for this > > > > > > > > > (SavepointTypeInformationFactory), and making it a > > > > > first-class, > > > > > > > > > config-selectable option would keep the catalog > > > > > > > forward-compatible > > > > > > > > as > > > > > > > > > serialization support grows. > > > > > > > > > 3. Robustness of the transparent decoding path. > > > > > > > > > - Related to (2): reconstructing values by mirroring > > the > > > > > binary > > > > > > > > > layout (PojoToRowDataDeserializer) is the most > powerful > > > but > > > > > > also > > > > > > > > the > > > > > > > > > most > > > > > > > > > fragile part of the design. How is it expected to > > behave > > > > > across > > > > > > > > > serializer > > > > > > > > > schema evolution / state migration (a serializer > > snapshot > > > > > that > > > > > > > > > differs from > > > > > > > > > the writer's), Kryo-fallback fields, nested/generic > > > types, > > > > > and > > > > > > > > > nullability? > > > > > > > > > - It would help to spell out the supported matrix and > > the > > > > > > failure > > > > > > > > > mode (hard error vs. degrade to raw bytes) up front, > > > since > > > > > this > > > > > > > > > is exactly > > > > > > > > > where "read without the user classes" is most likely > to > > > > break > > > > > > in > > > > > > > > > practice. > > > > > > > > > 4. Observability / summary reporting. > > > > > > > > > - The metadata view is a great start. Two small asks: > > > > > > > > > - per-subtask (or per-key-group) size granularity > in > > > > > > addition > > > > > > > to > > > > > > > > > per-operator, since skew is usually what you are > > > chasing > > > > > on > > > > > > > > > large state; > > > > > > > > > - optionally rounding out the size breakdown with > > > > > > managed/raw > > > > > > > > > operator state and channel state sizes for a full > > > > picture > > > > > > > > (noting > > > > > > > > > the > > > > > > > > > latter are in-flight / unaligned-checkpoint > buffers > > > > rather > > > > > > > > > than user state). > > > > > > > > > - A prominent upfront summary of the largest > operators > > / > > > > > state > > > > > > is > > > > > > > > > often what users want before drilling in. > > > > > > > > > > > > > > > > > > > > > > > > > > > Best regards, > > > > > > > > > Dennis > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/spaces/FLINK/pages/282102217/FLIP-398+Improve+Serialization+Configuration+And+Usage+In+Flink > > > > > > > > > [2] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/spaces/FLINK/pages/373886828/FLIP-538+Support+Custom+Generic+Type+Serializer > > > > > > > > > > > > > > > > > > On Mon, Jun 29, 2026 at 12:53 PM Gyula Fóra < > > [email protected] > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi Flink Devs! > > > > > > > > > > > > > > > > > > > > I would like to start the discussion about FLIP-599: > State > > > > > Catalog > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > State and stateful processing has always been one of the > > most > > > > > > > > fundamental > > > > > > > > > > features of Flink and a major contributor to its success > > and > > > > > global > > > > > > > > > > adoption. > > > > > > > > > > > > > > > > > > > > Over the years several apis and methods have been > developed > > > to > > > > > > > address > > > > > > > > > the > > > > > > > > > > need for external access and analytics such as the state > > > > > processor > > > > > > > > > > datastream / java apis, the since deprecated queryable > > state > > > > > > > > abstractions > > > > > > > > > > and more recently a number of table / SQL api connectors > to > > > > > access > > > > > > > > state > > > > > > > > > > metadata and keyed states in a somewhat limited way. > > > > > > > > > > > > > > > > > > > > Extending the current capabilities of the > > state-process-api, > > > > this > > > > > > > FLIP > > > > > > > > > aims > > > > > > > > > > to lift state processing, analytics and observability > to a > > > new > > > > > > level > > > > > > > > by > > > > > > > > > > introducing the State Catalog. > > > > > > > > > > > > > > > > > > > > State Catalog is a Flink SQL Catalog implementation that > > > allows > > > > > > > > > discovering > > > > > > > > > > savepoints/checkpoints and mapping their state > > automatically > > > to > > > > > SQL > > > > > > > > > tables. > > > > > > > > > > The tables are derived for the different operators and > > their > > > > > keyed > > > > > > > > states > > > > > > > > > > with schema matching the state structure. Most > importantly > > it > > > > > > > supports > > > > > > > > > > reading POJO / Avro and other structured and basic type > > > states > > > > > > > without > > > > > > > > > the > > > > > > > > > > original user classes (dependencies) by relying on > Flink's > > > > > > > transparent > > > > > > > > > and > > > > > > > > > > efficiently structured serializer formats. > > > > > > > > > > > > > > > > > > > > We have a fully functional prototype implementation > > developed > > > > > with > > > > > > > > Gabor > > > > > > > > > > Somogyi that we will be happy to share if the community > > > accepts > > > > > the > > > > > > > > > > proposal! > > > > > > > > > > > > > > > > > > > > Looking forward to your feedback and suggestions! > > > > > > > > > > > > > > > > > > > > Gyula > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/spaces/FLINK/pages/438009922/FLIP-599+State+Catalog > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
