[
https://issues.apache.org/jira/browse/BEAM-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16998390#comment-16998390
]
Alexey Romanenko commented on BEAM-8852:
----------------------------------------
Well, it would be great to write to multiple topics using EOS sink as well.
Afair, it won't be as easy as for normal sink and the main issue was related to
dynamic creation of ShardWriter depending on output topic in {{ProducerRecord}}
and write records into different topics in the same bundle (we can't shard by
topic/partition since we don't know all of them in advance).
We had a quick discussion about that some time ago on user@, please [take a
look|https://lists.apache.org/thread.html/cda1dd20c61648c279ca066a2ca3bd8357450bbe2744cd9fb28eb565%40%3Cuser.beam.apache.org%3E]
I'll be happy to discuss the possible ways of implementation if you have any.
> KafkaIO withEOS option does not use the topic provided in ProducerRecord
> ------------------------------------------------------------------------
>
> Key: BEAM-8852
> URL: https://issues.apache.org/jira/browse/BEAM-8852
> Project: Beam
> Issue Type: Bug
> Components: io-java-kafka
> Affects Versions: 2.15.0
> Reporter: DAVID MARTINEZ MATA
> Priority: Major
> Labels: eos, kafka
>
> Work In Progress.... defining the case accurately
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
