Hi Gustavo, Thank you for this very well written FLIP ! I really liked the examples, it helps understand the purpose well.
+1 for this proposal. One comment from my side. I understand there are only certain combinations that are allowed in `op_mapping`, especially in TO_CHANGELOG PTF. How do you plan to handle invalid cases? For example, what would happen if I specify: ["INSERT, DELETE", "ID"]. I suspect you'd fail during planning, am I right? Could we make it explicit in the FLIP? Best, Dawid On Fri, 13 Feb 2026 at 12:04, Timo Walther <[email protected]> wrote: > Hi Gustavo, > > thank you for this excellent design doc and coming up with all these > different combinations of use cases. These two PTFs will significantly > improve the Flink CDC story and help polishing the stream/table duality > story. > > +1 for this proposal. > > Naming-wise I could also imagine calling it TO_CDC/FROM_CDC. But the > current naming fits well to other API endpoints and Flink terminology, > which is why I support FROM_CHANGELOG/TO_CHANGELOG. > > Cheers, > Timo > > On 06.02.26 18:52, Gustavo de Morais wrote: > > Hi everyone, > > > > I'd like to propose FLIP-564: Support *FROM_CHANGELOG* and *TO_CHANGELOG* > > built-in PTFs [1] for discussion. > > > > Flink's DataStream API offers flexible methods like toChangelogStream() > and > > fromChangelogStream() to work with changelog streams, but SQL users > > currently lack this capability. This FLIP introduces two built-in Process > > Table Functions (PTFs) to bring *similar* functionality with additional > > features to Flink SQL: > > > > > > - *FROM_CHANGELOG*: Converts an append-only stream of CDC records > into a > > dynamic table, enabling, for example, custom CDC connector > implementations > > directly in SQL. > > - *TO_CHANGELOG*: Converts a dynamic table back into an append-only > > changelog stream - the first operator that makes it possible to > convert a > > retract/upsert stream back to append in SQL. > > > > > > Both PTFs support flexible operation mapping (e.g., Debezium-style 'c', > > 'u', 'd' codes), before/after image handling, configurable state TTL, and > > watermark-based ordering. They are designed to work symmetrically, so > > FROM_CHANGELOG(TO_CHANGELOG(table)) round-trips correctly. > > > > The naming follows the existing DataStream API convention. Alternative > > names were considered (e.g., ENCODE_CHANGELOG, DEMATERIALIZE) - they are > > under the rejected alternatives. If you have any better suggestions, feel > > free to share them here. > > > > Looking forward to your feedback and thoughts. > > > > Kind regards, > > Gustavo de Morais > > > > [1] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-564%3A+Support+FROM_CHANGELOG+and+TO_CHANGELOG+built-in+PTFs > > > >
