Kevin Lam created FLINK-36017:
---------------------------------
Summary: Support Passing Metadata to Formats in KafkaDynamicSink
Key: FLINK-36017
URL: https://issues.apache.org/jira/browse/FLINK-36017
Project: Flink
Issue Type: Improvement
Components: Connectors / Kafka, Formats (JSON, Avro, Parquet, ORC,
SequenceFile)
Affects Versions: 1.20.0
Reporter: Kevin Lam
Currently DynamicKafkaRecordSerializationSchema strips all [but the requested
physical
columns|https://github.com/apache/flink-connector-kafka/blob/86f796a01cba0d7b3adeb95f413e412c30e466f1/flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/table/DynamicKafkaRecordSerializationSchema.java#L100-L117]
and only passes projected physical rows to the formats and serializers.
This issue proposes supporting passing metadata from
[KafkaDynamicSink|https://github.com/apache/flink-connector-kafka/blob/86f796a01cba0d7b3adeb95f413e412c30e466f1/flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/table/KafkaDynamicSink.java#L68]
/ DynamicTableSink to the EncodingFormat, so that Formats can take the
metadata columns as input and process them.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
