[
https://issues.apache.org/jira/browse/BEAM-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16686983#comment-16686983
]
Alexey Romanenko commented on BEAM-6063:
----------------------------------------
Another way - we can keep {{Write}} using {{KV<K, V>}}, create new
{{WriteRecords}} transform, which will extend
{{PTransform<PCollection<ProducerRecord<K, V>>, PDone>}} and move all
functionality to this new transform. Old Write transform will be using new
{{WriteRecords}} under the hood. We can add new function {{writeRecords()}}
which will be used to write ProducerRecords directly, so we can benefit from
this for other features (headers, different topics). In this case, seems we
won't do any breaking changes to current API.
WDYT?
> KafkaIO: add writing support for ProducerRecord
> -----------------------------------------------
>
> Key: BEAM-6063
> URL: https://issues.apache.org/jira/browse/BEAM-6063
> Project: Beam
> Issue Type: Improvement
> Components: io-java-kafka
> Reporter: Alexey Romanenko
> Assignee: Alexey Romanenko
> Priority: Major
>
> Currently, the default input collection for {{KafkaIO.Write}} is
> {{PCollection<KV<K,V>>}}. To support writing of Kafka headers or different
> output Kafka topics, we need to change type of input collection to
> {{PCollection<ProducerRecord<K,V>>}}. Also, it will make sense to use
> {{ProducerRecord<K,V>}} instead of {{KV<K,V>}} internally in {{KafkaIO}} to
> keep all meta information.
> In the same time, we need to keep compatibility for old interface based on
> {{KV<K,V>}} but make it deprecated and totally move to
> {{ProducerRecord<K,V>}} later.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
