[
https://issues.apache.org/jira/browse/KAFKA-13348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Knowles Atchison Jr updated KAFKA-13348:
----------------------------------------
Description:
KAFKA-8586 added capture of Producer Exceptions which will kill the connector.
There is a need to allow the connector itself to be aware of these errors,
handle it in some manner, and continuing processing records.
The proposed change is to add a getter to the retry operator and allow producer
failure to be ignored via the "errors.tolerance" configuration setting. The
connect will receive a callback to commitRecord() with a null MetadataRecord.
The current behavior is kept if errors.tolerance is not set to all.
was:
KAFKA-8586 added capture of Producer Exceptions which will kill the connector.
There is a need to allow the connector itself to be aware of these errors,
handle it in some manner, and continuing processing records.
The proposed change is to add a function to SourceTask that allows handling of
the SourceRecord and Exception as thrown from the Producer. The SourceTask can
examine these items and determine if it is appropriate to die (current
behavior) or let the record be thrown away and continue processing.
The current behavior will be maintained by defaulting to returning false from
this function. If the implementing SourceTask override of this function returns
true, Kafka Connect will ignore this error record and continue processing.
> Allow Source Tasks to Handle Producer Exceptions
> ------------------------------------------------
>
> Key: KAFKA-13348
> URL: https://issues.apache.org/jira/browse/KAFKA-13348
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Affects Versions: 3.1.0, 2.8.1, 3.0.0
> Reporter: Knowles Atchison Jr
> Assignee: Knowles Atchison Jr
> Priority: Major
>
> KAFKA-8586 added capture of Producer Exceptions which will kill the connector.
> There is a need to allow the connector itself to be aware of these errors,
> handle it in some manner, and continuing processing records.
> The proposed change is to add a getter to the retry operator and allow
> producer failure to be ignored via the "errors.tolerance" configuration
> setting. The connect will receive a callback to commitRecord() with a null
> MetadataRecord.
> The current behavior is kept if errors.tolerance is not set to all.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
