[ 
https://issues.apache.org/jira/browse/CAMEL-26967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Claus Ibsen updated CAMEL-26967:
--------------------------------
    Fix Version/s: 4.24.0

> camel-kafka - breakOnFirstError with synchronous manual commits: a commitSync 
> timeout in the break skips the failed record and the rest of the poll
> ---------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-26967
>                 URL: https://issues.apache.org/jira/browse/CAMEL-26967
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-kafka
>            Reporter: shashank
>            Assignee: shashank
>            Priority: Major
>             Fix For: 4.24.0
>
>
> Found by the Kafka DST world of CAMEL-26947 (PR #29992, branch 
> {{camel-kafka-dst-world}}): with {{-Ddst.kafka.syncBreakOnFirstError=true}}, 
> seed 95 and 5 more of 1000 seeds fail the invariant "a committed offset never 
> passes a record the route has not finished with". Reproduced outside the DST 
> harness with a plain unit test (below).
> h3. What happens
> With {{breakOnFirstError=true}} and synchronous manual commits 
> ({{allowManualCommit=true}}, 
> {{kafkaManualCommitFactory=DefaultKafkaManualCommitFactory}}, so the 
> {{SyncCommitManager}}), a record that fails makes 
> {{KafkaRecordStreamingProcessor.processException}} commit the records routed 
> before it ({{commitManager.commit(partition)}}, a {{commitSync(offsets, 
> commitTimeoutMs)}}) and return the break, so {{KafkaFetchRecords}} reconnects 
> and the new Kafka consumer polls again from the committed offset.
> {{commitSync}} can throw: a {{TimeoutException}} when the timeout expires 
> before the commit completed (the caller does not know whether it was 
> applied), or {{CommitFailedException}} / {{RebalanceInProgressException}}. 
> The exception leaves {{processPolledRecords}} and reaches the catch of 
> {{KafkaFetchRecords.startPolling}}, which hands it to the poll exception 
> strategy. With {{pollOnError}} {{ERROR_HANDLER}} (the default), {{DISCARD}} 
> or {{RETRY}} the fetcher goes on polling with the same Kafka consumer, whose 
> position is already after the whole batch: the failed record and the records 
> after it in that poll are never routed, and the next manual commit (of a 
> later record) commits past them. Only {{RECONNECT}} (and {{STOP}}) avoid it.
> h3. Expected
> The option documentation of {{breakOnFirstError}}: "Using either the 
> SyncCommitManager or the AsyncCommitManager, the consumer will seek back to 
> the offset of the message that caused a failure, and then re-attempt to 
> process this message." {{KafkaConsumer#commitSync(Map, Duration)}} documents 
> the {{TimeoutException}} ("if the timeout expires before successful 
> completion of the offset commit") as a normal outcome. A failed commit in the 
> break must not turn the break into a skip: the consumer should still 
> reconnect and poll from the committed offset. If the commit was applied, that 
> is the failed record; if not, an earlier offset, and records are routed again 
> (at-least-once), never skipped.
> h3. Reproduction
> New {{KafkaConsumerBreakOnFirstErrorCommitTimeoutTest}} (a {{MockConsumer}} 
> on a log, a new Kafka consumer starts from the committed offset, as in 
> {{KafkaConsumerBreakOnFirstErrorPausedTest}}): records a..e, the route 
> commits each record manually, c fails once, the {{commitSync}} of the break 
> throws {{TimeoutException}} (not applied), and f is stored after it. On main 
> with {{pollOnError}} ERROR_HANDLER, DISCARD and RETRY the route gets {{[a, b, 
> f]}} and the group commits offset 6, so c, d and e are lost; RECONNECT 
> (control) gets all six.
> h3. Fix
> In {{processException}}, catch a {{KafkaException}} of the commit (not 
> {{WakeupException}} / {{InterruptException}}), report it to the exception 
> handler of the consumer, and return the break as before, so the consumer 
> reconnects and polls again from the committed offset. So that commit failure 
> no longer goes through {{pollOnError}} (it reached the route's error handler 
> through ERROR_HANDLER, and STOP stopped the consumer). Branch 
> {{camel-kafka-sync-commit-timeout-break-on-first-error}}.
> _Filed with Claude Code on behalf of allthingssecurity._



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to