[
https://issues.apache.org/jira/browse/CAMEL-27282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen resolved CAMEL-27282.
---------------------------------
Resolution: Fixed
Fixed by https://github.com/apache/camel/pull/30529 (merged to main as
a5de6b43e2d8), released in 4.24.0.
_Claude Code on behalf of davsclaus_
> camel-kafka - with offsetRepository a consumer that gets a partition back
> writes its old offset when the partition is revoked again
> -----------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-27282
> URL: https://issues.apache.org/jira/browse/CAMEL-27282
> Project: Camel
> Issue Type: Bug
> Components: camel-kafka
> Reporter: shashank
> Assignee: shashank
> Priority: Major
> Fix For: 4.24.0
>
>
> h3. What happens
> With {{offsetRepository}} (and no manual commits) each fetcher uses a
> {{CommitToOffsetManager}}. Its {{commit(partition)}} saves the last offset
> the fetcher processed in that partition but never removes it from its
> {{OffsetCache}} (the Sync and Async commit managers remove it after a
> commit). {{ClassicRebalanceListener.onPartitionsRevoked}} (and
> {{ResumeRebalanceListener}}) commits every revoked partition. The repository
> is shared by all the consumers of the endpoint ({{consumersCount}}) and can
> be shared by several applications.
> # Consumer A processes partition P up to offset 100 and saves it; a rebalance
> moves P to consumer B.
> # B resumes at 101, processes up to 500 and saves 500.
> # P comes back to A (B leaves, reconnects, scale down; several rebalances in
> a row are common during a rolling restart):
> {{OffsetPartitionAssignmentAdapter}} seeks to 501.
> # Another rebalance revokes P before A polled a record of it: the revoke
> saves A's cached 100. The repository goes back from 500 to 100, and the next
> owner processes 101..500 again.
> No message is lost, but every record processed since A last owned P is
> processed again.
> h3. How to reproduce
> {{CommitToOffsetManagerRevokeTest}}: a {{MockConsumer}}, a
> {{MemoryStateRepository}}, a {{CommitToOffsetManager}} and the
> {{ClassicRebalanceListener}}: save 100, revoke, set the repository to 500
> (the other consumer), assign (position 501), revoke: the repository holds 100.
> h3. Fix
> {{CommitToOffsetManager.commit(partition)}} removes the offset from its cache
> after saving it, as the other commit managers do.
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)