I'd like to propose introducing an upsert-kinesis connector for Flink SQL, following the same pattern as upsert-kafka (FLIP-149).
Currently, the kinesis connector only supports append (INSERT-only) streams. This means GROUP BY, deduplication, and streaming joins cannot write results to Kinesis Data Streams — the job fails with "Table sink doesn't support consuming update and delete changes." The proposal adds a separate 'upsert-kinesis' connector that: - Requires a PRIMARY KEY definition - Accepts INSERT, UPDATE_AFTER, and DELETE changelog events - Uses primary key fields as the Kinesis partition key - Writes empty-payload tombstones for deletes - Reuses the existing KinesisStreamsSink/Source (no code duplication) Full proposal: https://docs.google.com/document/d/1x5BmeUKC_Xi-2opdYUJmBZCtB9sj4DxPidvhy5PW4hE/edit?usp=sharing Draft implementation: https://github.com/apache/flink-connector-aws/pull/249 Looking forward to your feedback. Best regards, Francisco Morillo
