Yash Mayya created KAFKA-15145:
----------------------------------
Summary: AbstractWorkerSourceTask re-processes records filtered
out by SMTs on retriable exceptions
Key: KAFKA-15145
URL: https://issues.apache.org/jira/browse/KAFKA-15145
Project: Kafka
Issue Type: Bug
Components: KafkaConnect
Reporter: Yash Mayya
Assignee: Yash Mayya
If a RetriableException is thrown from an admin client or producer client
operation in
[AbstractWorkerSourceTask::sendRecords|https://github.com/apache/kafka/blob/5c2492bca71200806ccf776ea31639a90290d43e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L388],
the send operation is retried for the remaining records in the batch. There is
a minor bug in the logic for computing the remaining records for a batch which
causes records that are filtered out by the task's transformation chain to be
re-processed. This will also result in the SourceTask::commitRecord method
being called twice for the same record, which can cause certain types of source
connectors to fail. This bug seems to exist since when SMTs were first
introduced in 0.10.2
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
