[
https://issues.apache.org/jira/browse/FLINK-19149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Knauf updated FLINK-19149:
-------------------------------------
Description:
I would like to be able to interpret a compacted Kafka Topic as a upsert stream
in Apache Flink. Similarly, I would like to be able to write an upsert stream
to Kafka (into a compacted topic).
In both cases, the (possibly implicit) primary key of the Flink SQL Table would
need to correspond to the fields that make up the keys of the Kafka records.
A message for an existing key (with a higher offset) corresponds to an udate.
A message for an existing key with a null value is interpreted as a delete.
I would like to be able to interpret a compacted Kafka Topic as a versioned
table without creating an additional view (similar to Debezium/Canal; see
https://cwiki.apache.org/confluence/display/FLINK/FLIP-132+Temporal+Table+DDL+and+Temporal+Table+Join)
was:
I would like to be able to interpret a compacted Kafka Topic as a upsert stream
in Apache Flink. Similarly, I would like to be able to write an upsert stream
to Kafka (into a compacted topic).
I would like to be able to interpret a compacted Kafka Topic as a versioned
table without creating an additional view (similar to Debezium/Canal; see
https://cwiki.apache.org/confluence/display/FLINK/FLIP-132+Temporal+Table+DDL+and+Temporal+Table+Join)
In both cases, the (possibly implicit) primary key of the Flink SQL Table would
need to correspond to the fields that make up the keys of the Kafka records.
A message for an existing key (with a higher offset) corresponds to an udate.
A message for an existing key with a null value is interpreted as a delete.
> Compacted Kafka Topic can be interpreted as Changelog Stream
> ------------------------------------------------------------
>
> Key: FLINK-19149
> URL: https://issues.apache.org/jira/browse/FLINK-19149
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kafka
> Reporter: Konstantin Knauf
> Priority: Critical
>
> I would like to be able to interpret a compacted Kafka Topic as a upsert
> stream in Apache Flink. Similarly, I would like to be able to write an upsert
> stream to Kafka (into a compacted topic).
> In both cases, the (possibly implicit) primary key of the Flink SQL Table
> would need to correspond to the fields that make up the keys of the Kafka
> records.
> A message for an existing key (with a higher offset) corresponds to an udate.
> A message for an existing key with a null value is interpreted as a delete.
> I would like to be able to interpret a compacted Kafka Topic as a versioned
> table without creating an additional view (similar to Debezium/Canal; see
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-132+Temporal+Table+DDL+and+Temporal+Table+Join)
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
