[
https://issues.apache.org/jira/browse/CAMEL-26967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-26967 started by shashank.
----------------------------------------
> camel-kafka - breakOnFirstError with synchronous manual commits: a commitSync
> timeout in the break skips the failed record and the rest of the poll
> ---------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-26967
> URL: https://issues.apache.org/jira/browse/CAMEL-26967
> Project: Camel
> Issue Type: Bug
> Components: camel-kafka
> Reporter: shashank
> Assignee: shashank
> Priority: Major
>
> Found by the Kafka DST world of CAMEL-26947 (PR #29992, branch
> {{camel-kafka-dst-world}}): with {{-Ddst.kafka.syncBreakOnFirstError=true}},
> seed 95 and 5 more of 1000 seeds fail the invariant "a committed offset never
> passes a record the route has not finished with". Reproduced outside the DST
> harness with a plain unit test (below).
> h3. What happens
> With {{breakOnFirstError=true}} and synchronous manual commits
> ({{allowManualCommit=true}},
> {{kafkaManualCommitFactory=DefaultKafkaManualCommitFactory}}, so the
> {{SyncCommitManager}}), a record that fails makes
> {{KafkaRecordStreamingProcessor.processException}} commit the records routed
> before it ({{commitManager.commit(partition)}}, a {{commitSync(offsets,
> commitTimeoutMs)}}) and return the break, so {{KafkaFetchRecords}} reconnects
> and the new Kafka consumer polls again from the committed offset.
> {{commitSync}} can throw: a {{TimeoutException}} when the timeout expires
> before the commit completed (the caller does not know whether it was
> applied), or {{CommitFailedException}} / {{RebalanceInProgressException}}.
> The exception leaves {{processPolledRecords}} and reaches the catch of
> {{KafkaFetchRecords.startPolling}}, which hands it to the poll exception
> strategy. With {{pollOnError}} {{ERROR_HANDLER}} (the default), {{DISCARD}}
> or {{RETRY}} the fetcher goes on polling with the same Kafka consumer, whose
> position is already after the whole batch: the failed record and the records
> after it in that poll are never routed, and the next manual commit (of a
> later record) commits past them. Only {{RECONNECT}} (and {{STOP}}) avoid it.
> h3. Expected
> The option documentation of {{breakOnFirstError}}: "Using either the
> SyncCommitManager or the AsyncCommitManager, the consumer will seek back to
> the offset of the message that caused a failure, and then re-attempt to
> process this message." {{KafkaConsumer#commitSync(Map, Duration)}} documents
> the {{TimeoutException}} ("if the timeout expires before successful
> completion of the offset commit") as a normal outcome. A failed commit in the
> break must not turn the break into a skip: the consumer should still
> reconnect and poll from the committed offset. If the commit was applied, that
> is the failed record; if not, an earlier offset, and records are routed again
> (at-least-once), never skipped.
> h3. Reproduction
> New {{KafkaConsumerBreakOnFirstErrorCommitTimeoutTest}} (a {{MockConsumer}}
> on a log, a new Kafka consumer starts from the committed offset, as in
> {{KafkaConsumerBreakOnFirstErrorPausedTest}}): records a..e, the route
> commits each record manually, c fails once, the {{commitSync}} of the break
> throws {{TimeoutException}} (not applied), and f is stored after it. On main
> with {{pollOnError}} ERROR_HANDLER, DISCARD and RETRY the route gets {{[a, b,
> f]}} and the group commits offset 6, so c, d and e are lost; RECONNECT
> (control) gets all six.
> h3. Fix
> In {{processException}}, catch a {{KafkaException}} of the commit (not
> {{WakeupException}} / {{InterruptException}}), report it to the exception
> handler of the consumer, and return the break as before, so the consumer
> reconnects and polls again from the committed offset. So that commit failure
> no longer goes through {{pollOnError}} (it reached the route's error handler
> through ERROR_HANDLER, and STOP stopped the consumer). Branch
> {{camel-kafka-sync-commit-timeout-break-on-first-error}}.
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)