[
https://issues.apache.org/jira/browse/KAFKA-13185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18101167#comment-18101167
]
Yunseop Eom commented on KAFKA-13185:
-------------------------------------
PR opened: https://github.com/apache/kafka/pull/23048 for KAFKA-13185. Fixed
the Kafka Connect sink-task recovery path so a failed preCommit rewind clears
both the pending messageBatch and its associated origOffsets before the next
poll, preventing stale records and rewound offsets from being delivered again.
Added a focused regression test using a retriable put followed by a failed
preCommit; the test was RED before the fix and GREEN after it.
WorkerSinkTaskTest, connect:runtime Spotless, Checkstyle, SpotBugs, and the
full connect:runtime test task passed. Status: PR open and awaiting review.
> Kafka Connect should clear messageBatch after rewind
> ----------------------------------------------------
>
> Key: KAFKA-13185
> URL: https://issues.apache.org/jira/browse/KAFKA-13185
> Project: Kafka
> Issue Type: Bug
> Components: connect
> Reporter: Jiri Syrovy
> Priority: Critical
>
> WorkerSinkTask contains logic to handle RetriableException that happened in
> either put or flush. If the same Exception happens for longer than poll or
> timeout interval and it's thrown as well in preCommit/flush it causes offset
> to rewind back to the last committed position. Meanwhile messageBatch
> (message buffer) is not cleared and during the next poll it's used passed to
> the put method.
> During the next poll the same message is read from the broker and again
> passed to the put method.
> This results into duplicated message sent by the connector after
> RetrieableException.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
