[
https://issues.apache.org/jira/browse/KAFKA-6490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354959#comment-16354959
]
Ewen Cheslack-Postava commented on KAFKA-6490:
----------------------------------------------
[~prasanna1433] I've given you wiki permissions, you should be able to create a
page now.
The dead letter queue is something I've specifically heard from a number of
users, so it's definitely in demand. The list I gave is based on a ton of real
user feedback, so I feel pretty confident that it is both a) covering important
use cases and b) comprehensive enough to address the vast majority of use
cases. But I'm of course open to discussion of the options. I suspect *more*
options would be the result rather than removing some. If you want to take on
this improvement, we can of course discuss further in the KIP thread :)
With respect to the version, new features should almost universally be worked
on in trunk – older release branches are reserved for bug fixes. In this case,
since we just cut 1.1 branches, this would be a candidate for 1.2 / 2.0 and can
simply be developed against trunk.
> JSON SerializationException Stops Connect
> -----------------------------------------
>
> Key: KAFKA-6490
> URL: https://issues.apache.org/jira/browse/KAFKA-6490
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 1.0.0
> Reporter: William R. Speirs
> Priority: Major
> Attachments: KAFKA-6490_v1.patch
>
>
> If you configure KafkaConnect to parse JSON messages, and you send it a
> non-JSON message, the SerializationException message will bubble up to the
> top, and stop KafkaConnect. While I understand sending non-JSON to a JSON
> serializer is a bad idea, I think that a single malformed message stopping
> all of KafkaConnect is even worse.
> The data exception is thrown here:
> [https://github.com/apache/kafka/blob/trunk/connect/json/src/main/java/org/apache/kafka/connect/json/JsonConverter.java#L305]
>
> From the call here:
> [https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java#L476]
> This bubbles all the way up to the top, and KafkaConnect simply stops with
> the message: {{ERROR WorkerSinkTask\{id=elasticsearch-sink-0} Task threw an
> uncaught and unrecoverable exception
> (org.apache.kafka.connect.runtime.WorkerTask:172)}}
> Thoughts on adding a {{try/catch}} around the {{for}} loop in
> WorkerSinkTask's {{convertMessages}} so messages that don't properly parse
> are logged, but simply ignored? This way KafkaConnect can keep working even
> when it encounters a message it cannot decode?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
