[
https://issues.apache.org/jira/browse/BEAM-52?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264860#comment-15264860
]
Raghu Angadi commented on BEAM-52:
----------------------------------
Ah, thanks. just skimmed through it. It looks pretty much on the same lines as
what I have. couple of differences :
- Kafka producer uses the key for selecting a partition. I wanted to retain
that functionality for users. So I apply our coders inside custom Kafka
serializers. Otherwise Kafka will hash on serialized byte array.
- I noticed you are catching the exceptions in a callback and reporting it
back.. may be I should do that too. will get Dan's opinion as well in PR.
I will ping you to review my pull request.
> KafkaIO - bounded/unbounded, source/sink
> ----------------------------------------
>
> Key: BEAM-52
> URL: https://issues.apache.org/jira/browse/BEAM-52
> Project: Beam
> Issue Type: New Feature
> Components: sdk-java-extensions
> Reporter: Daniel Halperin
> Assignee: Raghu Angadi
>
> We should support Apache Kafka. The priority list is probably:
> * UnboundedSource
> * unbounded Sink
> * BoundedSource
> * bounded Sink
> The connector should be well-tested, especially around UnboundedSource
> checkpointing and resuming, and data duplication.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
