Lorenzo Benvenuti created CAMEL-18759:
-----------------------------------------
Summary: When Kafka pausable consumer is resumed, it reads all the
messages from the earliest offset
Key: CAMEL-18759
URL: https://issues.apache.org/jira/browse/CAMEL-18759
Project: Camel
Issue Type: Bug
Reporter: Lorenzo Benvenuti
Hi,
I'm trying to add the circuit breaker pattern to a Kafka consumer. In short, I
want to suspend the consumer when an exception happens due to an outage in
downstream systems and resume the consumer when downstream systems are up
again. As a reference I used this example from the Camel documentation:
https://camel.apache.org/components/3.18.x/eips/resume-strategies.html#_pausable_consumers_api
(coming from this test:
https://github.com/apache/camel/blob/2c01082032346753cc6621ab4530b0969f12bac7/components/camel-kafka/src/test/java/org/apache/camel/component/kafka/integration/pause/KafkaPausableConsumerCircuitBreakerIT.java)
However, I noticed that every time the consumer is resumed it starts reading
the messages from the earliest offset, and this causes a redelivery of all the
messages stored in the partition; as far as I can tell it happens because
{{KafkaConsumerListener#afterProcess}} invokes {{consumer.seekToBeginning}}.
I've created a reproducer here:
https://github.com/lorenzobenvenuti/camel-kafka-pausable-bug
Thanks,
lorenzo
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
