Greg Harris created KAFKA-16069:
-----------------------------------
Summary: Source Tasks re-transform records after Retriable
exceptions
Key: KAFKA-16069
URL: https://issues.apache.org/jira/browse/KAFKA-16069
Project: Kafka
Issue Type: Bug
Components: KafkaConnect
Reporter: Greg Harris
In the SinkTask, records which fail to be delivered to the task#put with a
Retriable exception are re-delivered on the next iteration. The SourceTask
follows a similar pattern, where records which fail to be delivered to
Producer#send with a Retriable exception are retried.
However, the SinkTask accumulates the post-transform records, and does not
recompute the transformations over again after a retriable exception. The
SourceTask does not accumulate ProducerRecords, and instead recomputes the
transformations and converters starting from the pre-transformation
AbstractWorkerSourceTask#toSend list.
This means that stateful transformations and converters may see the records
rewind, without any indication that the records are the same. For stateless
transformations and converters, this means that redundant computation is
performed that may be better allocated to other tasks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
