[
https://issues.apache.org/jira/browse/CAMEL-20044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17783355#comment-17783355
]
Mike Barlotta commented on CAMEL-20044:
---------------------------------------
In the _KafkaRecordProcessor_ it seems that using _record.offset_ for the
forceCommit is avoiding the problem in this issue (line #158 in PR)
* note: can still result in the _lastResult_ being wrong but now we are not
using it to force a commit
** when _lastResult_ is wrong then the message with the error is tried more
than 1x
> camel-kafka - On rejoining consumer group Camel can set offset incorrectly
> causing messages to be replayed
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-20044
> URL: https://issues.apache.org/jira/browse/CAMEL-20044
> Project: Camel
> Issue Type: Bug
> Components: camel-kafka
> Affects Versions: 3.21.0
> Environment: * Rocky Linux 8.7
> * Open JDK 11.0.8
> * Camel 3.21.0
> * Spring Boot 2.7.14
> * Strimzi Kafka 0.28.0/3.0.0
> Reporter: Mike Barlotta
> Priority: Major
> Attachments: camel-kafka-offset.11-06-2023.log
>
>
> {*}Reproducing (intermittent){*}:
> * Configure camel kafka consumer with following:
> ** autoCommitEnable = false
> ** allowManualCommit = true
> ** autoOffsetReset = earliest
> ** maxPollRecords = 1
> ** breakOnFirstError = true
> * Produce a series of records to kafka record to both partitions.
> * Throw an exception that is unhandled
> * commit the offset in the onException block
> *Expected behavior:*
> * Application should consume the record 1 more time, then move on to the
> next offset in the partition
> *Actual behavior:*
> * Application will often work. Occasionally will use the offset from another
> partition and assign that to the partition where the record failed. This can
> then result in the consumer replaying messages instead of moving forward.
> I put together a sample that can recreate the error. However, given its
> intermittent nature it may not fail on each run. I have included the logs
> from 3 different runs on my laptop from this test. Two of them show the error
> occurring. One of the them has a successful run. I have also provided more
> details in the README.
> * [https://github.com/CodeSmell/CamelKafkaOffset]
> This seems related to other issues with how Camel processes the
> _breakOnFirstError_ attribute.
> * CAMEL-14935
> * CAMEL-17925
> * CAMEL-18350
> * CAMEL-19894
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
