Hey Hao, Great, thanks and thanks for the feedback everyone. I think we can now move on to the vote.
Kind regards Gustavo On Thu, 26 Feb 2026 at 21:02, Hao Li via dev <[email protected]> wrote: > Hi Gustavo, > > The config and fail as default sounds good! > > Thanks, > Hao > > On Thu, Feb 26, 2026 at 3:13 AM Gustavo de Morais <[email protected]> > wrote: > > > Thanks for the feedback, Leonard and Hao! > > > > That's a valid point regarding invalid op codes, Hao. We could add an > > invalid_op_handling parameter to the PTFs with three options: 'FAIL', > > 'LOG', or 'SKIP'. It would be an additional parameter and look like this: > > > > ... > > invalid_op_handling => 'FAIL' | 'LOG' | 'SKIP' > > ... > > > > DLQ would be a way higher LOE to implement so I'd say we skip that option > > for now. I'd actually lean toward making FAIL as the default since it's a > > serious issue and should be addressed explicitly. Users can then change > it > > if necessary. > > > > Kind regards > > Gustavo > > > > On Thu, 26 Feb 2026 at 07:23, Hao Li via dev <[email protected]> > wrote: > > > > > Thanks Gustavo! +1 to the proposal. > > > > > > Only question I have is error handling: looks it's proposed to ignore > > > invalid ops in `ops_mapping` and log warning. I'm wondering if we can > > make > > > it configurable that it can also fail or log to DLQ? Ignore silently > make > > > it hard to reason what's going on and if all records have invalid ops, > it > > > would be lots of warnings. > > > > > > Thanks, > > > Hao > > > > > > On Wed, Feb 25, 2026 at 7:18 AM Leonard Xu <[email protected]> wrote: > > > > > > > +1 for the proposal, It will help Flink CDC users a lot especially > for > > > SQL > > > > user cases. > > > > > > > > The naming FROM_CHANGELOG/TO_CHANGELOG are aligned with DataStream > API > > > > style which makes sense to me. > > > > > > > > Best, > > > > Leonard > > > > > > > > On Thu, Feb 19, 2026 at 12:52 AM Gustavo de Morais < > > > [email protected] > > > > > > > > > wrote: > > > > > > > > > Hi Timo and Dawid, > > > > > > > > > > Thanks for taking a look and the feedback! To address your valid > > point, > > > > > I've included a new section called "4.3.6 Invalid op_mapping > > > > Combinations", > > > > > Dawid. Apart from that, I've also included one functionality to > both > > > > > functions called "passthrough". It allows users to carry top-level > > > fields > > > > > (e.g., ts_ms) through the PTF when before/after are used, instead > of > > > > > dropping them. It seems to be a common use and a useful additional > > > > > functionality. Documentation and use cases were added. > > > > > > > > > > If there are no further comments, I'd like to start the vote in the > > > > coming > > > > > days. > > > > > > > > > > Kind regards, > > > > > Gustavo > > > > > > > > > > On Fri, 13 Feb 2026 at 12:13, Dawid Wysakowicz < > > [email protected] > > > > > > > > > wrote: > > > > > > > > > > > Hi Gustavo, > > > > > > Thank you for this very well written FLIP ! I really liked the > > > > examples, > > > > > it > > > > > > helps understand the purpose well. > > > > > > > > > > > > +1 for this proposal. > > > > > > > > > > > > One comment from my side. I understand there are only certain > > > > > combinations > > > > > > that are allowed in `op_mapping`, especially in TO_CHANGELOG PTF. > > How > > > > do > > > > > > you plan to handle invalid cases? For example, what would happen > > if I > > > > > > specify: ["INSERT, DELETE", "ID"]. I suspect you'd fail during > > > > planning, > > > > > > am I right? Could we make it explicit in the FLIP? > > > > > > > > > > > > Best, > > > > > > Dawid > > > > > > > > > > > > On Fri, 13 Feb 2026 at 12:04, Timo Walther <[email protected]> > > > wrote: > > > > > > > > > > > > > Hi Gustavo, > > > > > > > > > > > > > > thank you for this excellent design doc and coming up with all > > > these > > > > > > > different combinations of use cases. These two PTFs will > > > > significantly > > > > > > > improve the Flink CDC story and help polishing the stream/table > > > > duality > > > > > > > story. > > > > > > > > > > > > > > +1 for this proposal. > > > > > > > > > > > > > > Naming-wise I could also imagine calling it TO_CDC/FROM_CDC. > But > > > the > > > > > > > current naming fits well to other API endpoints and Flink > > > > terminology, > > > > > > > which is why I support FROM_CHANGELOG/TO_CHANGELOG. > > > > > > > > > > > > > > Cheers, > > > > > > > Timo > > > > > > > > > > > > > > On 06.02.26 18:52, Gustavo de Morais wrote: > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > I'd like to propose FLIP-564: Support *FROM_CHANGELOG* and > > > > > > *TO_CHANGELOG* > > > > > > > > built-in PTFs [1] for discussion. > > > > > > > > > > > > > > > > Flink's DataStream API offers flexible methods like > > > > > toChangelogStream() > > > > > > > and > > > > > > > > fromChangelogStream() to work with changelog streams, but SQL > > > users > > > > > > > > currently lack this capability. This FLIP introduces two > > built-in > > > > > > Process > > > > > > > > Table Functions (PTFs) to bring *similar* functionality with > > > > > additional > > > > > > > > features to Flink SQL: > > > > > > > > > > > > > > > > > > > > > > > > - *FROM_CHANGELOG*: Converts an append-only stream of CDC > > > > records > > > > > > > into a > > > > > > > > dynamic table, enabling, for example, custom CDC > connector > > > > > > > implementations > > > > > > > > directly in SQL. > > > > > > > > - *TO_CHANGELOG*: Converts a dynamic table back into an > > > > > append-only > > > > > > > > changelog stream - the first operator that makes it > > possible > > > to > > > > > > > convert a > > > > > > > > retract/upsert stream back to append in SQL. > > > > > > > > > > > > > > > > > > > > > > > > Both PTFs support flexible operation mapping (e.g., > > > Debezium-style > > > > > 'c', > > > > > > > > 'u', 'd' codes), before/after image handling, configurable > > state > > > > TTL, > > > > > > and > > > > > > > > watermark-based ordering. They are designed to work > > > symmetrically, > > > > so > > > > > > > > FROM_CHANGELOG(TO_CHANGELOG(table)) round-trips correctly. > > > > > > > > > > > > > > > > The naming follows the existing DataStream API convention. > > > > > Alternative > > > > > > > > names were considered (e.g., ENCODE_CHANGELOG, > DEMATERIALIZE) - > > > > they > > > > > > are > > > > > > > > under the rejected alternatives. If you have any better > > > > suggestions, > > > > > > feel > > > > > > > > free to share them here. > > > > > > > > > > > > > > > > Looking forward to your feedback and thoughts. > > > > > > > > > > > > > > > > Kind regards, > > > > > > > > Gustavo de Morais > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-564%3A+Support+FROM_CHANGELOG+and+TO_CHANGELOG+built-in+PTFs > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
