[
https://issues.apache.org/jira/browse/KAFKA-9982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17111563#comment-17111563
]
Chris Egerton edited comment on KAFKA-9982 at 5/20/20, 2:43 AM:
----------------------------------------------------------------
[~q.xu] I'd like to mark this closed, but if you've encountered problems with
at-least-once delivery in the framework please feel free to reopen with steps
to reproduce. However, from what I can tell the framework does provide this
guarantee correctly, or at least, it doesn't have the issues described in this
ticket.
was (Author: chrisegerton):
[~q.xu] I'd like to mark this closed, but if you've encountered problems with
at-least-once delivery in the framework please feel free to reopen with steps
to reproduce However, from what I can tell the framework does provide this
guarantee correctly, or at least, it doesn't have the issues described in this
ticket.
> [kafka-connect] Source connector does not guarantee at least once delivery
> --------------------------------------------------------------------------
>
> Key: KAFKA-9982
> URL: https://issues.apache.org/jira/browse/KAFKA-9982
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 2.5.0
> Reporter: Qinghui Xu
> Priority: Major
>
> In kafka-connect runtime, the WorkerSourceTask is responsible for sending
> records to the destination topics and managing the source offset commit.
> Committed offsets are then used later for recovery of tasks during rebalance
> or restart.
> But there are two concerns when looking into the WorkerSourceTask
> implementation:
> * When producer fail to send records, there's no retry but just skipping
> offset commit and then execute next loop (poll for new records)
> * The offset commit and effectively sending records over network are in fact
> asynchronous, which means the offset commit could happen before records are
> received by brokers, and a rebalance/restart in this gap could lead to
> message loss.
> The conclusion is thus that the source connector does not support at least
> once semantics by default (without the plugin implementation making extra
> effort itself). I consider this as a bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
