[
https://issues.apache.org/jira/browse/CAMEL-14980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305501#comment-17305501
]
Claus Ibsen edited comment on CAMEL-14980 at 3/20/21, 4:56 PM:
---------------------------------------------------------------
[~cless91] thanks for the reproducer - With the PR then the serialize exception
causes Camel to not retry and you can bridge to Camel's error handler (or not).
And the kafka consumer is closed and terminated.
I assume that in any kind of polling exception we should make camel-kafka
consumer be able to do
1) retry poll again (may fail forever, eg poison message)
2) fail and terminate (consumer is stopped/died - not sure if this is an ideal
use-case)
3) if bridge error handler, then let camel error handler handle it, but keep
consumer up to pickup next message
And for all of this you can decide what to do per exception - so let some
retry, and all others do 2 or 3.
was (Author: davsclaus):
[~cless91] thanks for the reproducer - With the PR then the serialize exception
causes Camel to not retry and you can bridge to Camel's error handler (or not).
And the kafka consumer is closed and terminated.
> camel-kafka - SerializationException - consumer keeps leaving and rejoining
> the group
> -------------------------------------------------------------------------------------
>
> Key: CAMEL-14980
> URL: https://issues.apache.org/jira/browse/CAMEL-14980
> Project: Camel
> Issue Type: Bug
> Components: camel-kafka
> Affects Versions: 3.2.0
> Reporter: joseph m'bimbi-bene
> Assignee: Claus Ibsen
> Priority: Major
> Fix For: 3.9.0
>
> Attachments: camel-kafka-errors.txt, poc_camel_kafka.tar.gz
>
>
> Hello everyone,
>
> I found out i few days ago that if a `SerializationException` is thrown when
> the consumer tries to poll messages, it will keep leaving and joining the
> consumer-group indefinitely and without any informative log.
> The exception cannot either be handled by any camel exception handler.
> After some searching in the code i found out the culprit:
> {code:java}
> // org.apache.camel.component.kafka.KafkaConsumer (ligns 406-415):
> catch (KafkaException e) {
> // some kind of error in kafka, it may happen during
> // unsubscribing or during normal processing
> if (unsubscribing){
> getExceptionHandler().handleException("Error unsubscribing " + threadId +
> " from kafka topic " + topicName, e);
> }else {
> LOG.debug("KafkaException consuming {} from topic {} causedby {}. Will
> attempt to re-connect on next run", threadId, topicName, e.getMessage());
> reConnect = true;
> }
> }
> {code}
>
> `SerializationException` extends from `KafkaException`, but it is definitely
> not a recoverable exception.
> It logs with debug level, which makes it hard to track, there are SO many
> things logging in debug.
> It it cannot be handled by any camel exception handling mechanism.
> I think it would be better to either:
> - change that catch so that it pinpoints the subclasses of `KafkaException`
> that are actually recoverable from rejoining (maybe `WakeupException` and a
> couple others)
> - add a `catch` block for `SerializationException` and maybe
> `ConfigException` and `OAuthBearerConfigException` before, with a log error
> andallow the user to handle those exceptions
> - remove that catch block entirely and let users handle any KafkaException
> however they see fit.
> Thank you
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
