[
https://issues.apache.org/jira/browse/BEAM-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Romanenko updated BEAM-6063:
-----------------------------------
Description:
Currently, the default input collection for {{KafkaIO.Write}} is
{{PCollection<KV<K, V>>}}. To support writing of Kafka headers or different
output Kafka topics, we need to change type of input collection to
{{PCollection<ProducerRecord<K, V>>}}. Also, it will make sense to use
{{ProducerRecord<K, V>}} instead of {{KV<K ,V>}} internally in {{KafkaIO}} to
keep all meta information.
In the same time, we need to keep compatibility for old interface based on
{{KV<K, V>}} but make it deprecated and totally move to {{ProducerRecord<K,
V>}} later.
was:
Currently, the default input collection for KafkaIO.Write is
{{PCollection<KV<K, V>>}}. To support writing of Kafka headers or different
output Kafka topics, we need to provide change input collection to
{{PCollection<KV<K, V>>}}
KafkaIO.writeRecords() which is a PCollection<ProducerRecord<K, V>, PDone>
{{KafkaIO.writeRecords()}} which is a {{PCollection<ProducerRecord<K, V>,
PDone>}}?
> KafkaIO: add support of ProducerRecord for writing
> ---------------------------------------------------
>
> Key: BEAM-6063
> URL: https://issues.apache.org/jira/browse/BEAM-6063
> Project: Beam
> Issue Type: Improvement
> Components: io-java-kafka
> Reporter: Alexey Romanenko
> Assignee: Raghu Angadi
> Priority: Major
>
> Currently, the default input collection for {{KafkaIO.Write}} is
> {{PCollection<KV<K, V>>}}. To support writing of Kafka headers or different
> output Kafka topics, we need to change type of input collection to
> {{PCollection<ProducerRecord<K, V>>}}. Also, it will make sense to use
> {{ProducerRecord<K, V>}} instead of {{KV<K ,V>}} internally in {{KafkaIO}} to
> keep all meta information.
> In the same time, we need to keep compatibility for old interface based on
> {{KV<K, V>}} but make it deprecated and totally move to {{ProducerRecord<K,
> V>}} later.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
