Hi Yanquan, Thanks for the question, please see the FLIP content related this.
---BEGIN FLIP CONTENT--- The target of this implementation proposal is to provide SQL support for keyed states. Other types are going to be covered in further FLIP documents. ---END FLIP CONTENT--- Keyed state data which is defined by users in for example process functions, operator or non-keyed state data is defined by internal Flink operators for example sources/sinks. In short, we're intended to add this support later on which means that further configuration parameters needs to be added. BR, G On Thu, Dec 19, 2024 at 2:12 PM Yanquan Lv <decq12y...@gmail.com> wrote: > Hi, Gabor. Thanks for driving this FLIP and this immediately reminded me > of exploring the status of the Source connector like Kafka or MySQLCDC to > understand the progress of reading information. > > In SourceOperator, the state of reader in contained in a ListState named > `SourceReaderState`[1], however, the state type of `SourceReaderState` is > `byte[]`, so I am worried that we can't get any understandable information > using this connector. Do we consider supporting this scenario？ > > If we want to support this scenario, perhaps we need to add more > parameters like fields.#.state-version-serializer and > fields.#.state-version to To specify the > org.apache.flink.core.io.SimpleVersionedSerializer[2] > and version to be used, And it also requires more complex implementation. > > Do I have any misunderstandings? Is this requirement something we need to > support, or will it need support in the future. > > > [1] > https://github.com/apache/flink/blob/9e38f32889e097f5a9b81f949643a2b037c1035c/flink-runtime/src/main/java/org/apache/flink/streaming/api/operators/SourceOperator.java#L109 > [2] > https://github.com/apache/flink/blob/9e38f32889e097f5a9b81f949643a2b037c1035c/flink-core/src/main/java/org/apache/flink/core/io/SimpleVersionedSerializer.java#L50 > > > > > > > 2024年12月18日 17:43，Gabor Somogyi <gabor.g.somo...@gmail.com> 写道： > > > > Hi All, > > > > I'd like to start a discussion of FLIP-496: SQL connector for keyed state > > data [1]. > > Feel free to add your thoughts to make this feature better. > > > > [1] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-496%3A+SQL+connector+for+keyed+state+data > > > > BR, > > G > >
