[
https://issues.apache.org/jira/browse/KAFKA-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14995343#comment-14995343
]
ASF GitHub Bot commented on KAFKA-2480:
---------------------------------------
GitHub user ewencp opened a pull request:
https://github.com/apache/kafka/pull/450
KAFKA-2480: Handle retriable and non-retriable exceptions thrown by sink
tasks.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ewencp/kafka
kafka-2480-unrecoverable-task-errors
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/450.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #450
----
commit f273295444c40add45dfac24038e52313f70af7f
Author: Ewen Cheslack-Postava <m...@ewencp.org>
Date: 2015-11-06T23:18:30Z
KAFKA-2480: Handle retriable and non-retriable exceptions thrown by sink
tasks.
----
> Handle non-CopycatExceptions from SinkTasks
> -------------------------------------------
>
> Key: KAFKA-2480
> URL: https://issues.apache.org/jira/browse/KAFKA-2480
> Project: Kafka
> Issue Type: Sub-task
> Components: copycat
> Reporter: Ewen Cheslack-Postava
> Assignee: Ewen Cheslack-Postava
> Fix For: 0.9.0.0
>
>
> Currently we catch Throwable in WorkerSinkTask, but we just log the
> exception. This can lead to data loss because it indicates the messages in
> the {{put(records)}} call probably were not handled properly. We need to
> decide what the policy for handling these types of exceptions should be --
> try repeating the same records again, risking duplication? or skip them,
> risking loss? or kill the task immediately and require intervention since
> it's unclear what happened?
> SourceTasks don't have the same concern -- they can throw other exceptions
> and as long as we catch them, it is up to the connector to ensure that it
> does not lose data as a result.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
