Gustavo de Morais created FLINK-39258:
-----------------------------------------
Summary: FLIP-564: Support FROM_CHANGELOG and TO_CHANGELOG
built-in PTFs
Key: FLINK-39258
URL: https://issues.apache.org/jira/browse/FLINK-39258
Project: Flink
Issue Type: New Feature
Reporter: Gustavo de Morais
Assignee: Gustavo de Morais
Flink's DataStream API offers flexible methods like toChangelogStream() and
fromChangelogStream() to work with changelog streams, but SQL users currently
lack this capability.
This FLIP introduces two built-in Process Table Functions (PTFs) to bring
_similar_ functionality with additional features to Flink SQL:
- {*}FROM_CHANGELOG{*}: Converts an append-only stream of CDC records into a
dynamic table, enabling, for example, custom CDC connector implementations
directly in SQL. - {*}TO_CHANGELOG{*}: Converts a dynamic table back into an
append-only changelog stream - the first operator that makes it possible to
convert a retract/upsert stream back to append in SQL.
Both PTFs will eventually support flexible operation mapping (e.g.,
Debezium-style 'c', 'u', 'd' codes), before/after image handling, configurable
state TTL, and watermark-based ordering. They are designed to work
symmetrically, so FROM_CHANGELOG(TO_CHANGELOG(table)) round-trips correctly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
