[
https://issues.apache.org/jira/browse/KAFKA-9982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120173#comment-17120173
]
Qinghui Xu commented on KAFKA-9982:
-----------------------------------
Thanks [~ChrisEgerton], very nice pointers, now it starts to make sense for me.
But I still have doubts on some scenarios, I'll try to make some tests to
check. Will keep you posted about it.
> [kafka-connect] Source connector does not guarantee at least once delivery
> --------------------------------------------------------------------------
>
> Key: KAFKA-9982
> URL: https://issues.apache.org/jira/browse/KAFKA-9982
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 2.5.0
> Reporter: Qinghui Xu
> Priority: Major
>
> In kafka-connect runtime, the WorkerSourceTask is responsible for sending
> records to the destination topics and managing the source offset commit.
> Committed offsets are then used later for recovery of tasks during rebalance
> or restart.
> But there are two concerns when looking into the WorkerSourceTask
> implementation:
> * When producer fail to send records, there's no retry but just skipping
> offset commit and then execute next loop (poll for new records)
> * The offset commit and effectively sending records over network are in fact
> asynchronous, which means the offset commit could happen before records are
> received by brokers, and a rebalance/restart in this gap could lead to
> message loss.
> The conclusion is thus that the source connector does not support at least
> once semantics by default (without the plugin implementation making extra
> effort itself). I consider this as a bug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
