[ 
https://issues.apache.org/jira/browse/CAMEL-27282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Claus Ibsen resolved CAMEL-27282.
---------------------------------
    Resolution: Fixed

Fixed by https://github.com/apache/camel/pull/30529 (merged to main as 
a5de6b43e2d8), released in 4.24.0.

_Claude Code on behalf of davsclaus_

> camel-kafka - with offsetRepository a consumer that gets a partition back 
> writes its old offset when the partition is revoked again
> -----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-27282
>                 URL: https://issues.apache.org/jira/browse/CAMEL-27282
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-kafka
>            Reporter: shashank
>            Assignee: shashank
>            Priority: Major
>             Fix For: 4.24.0
>
>
> h3. What happens
> With {{offsetRepository}} (and no manual commits) each fetcher uses a 
> {{CommitToOffsetManager}}. Its {{commit(partition)}} saves the last offset 
> the fetcher processed in that partition but never removes it from its 
> {{OffsetCache}} (the Sync and Async commit managers remove it after a 
> commit). {{ClassicRebalanceListener.onPartitionsRevoked}} (and 
> {{ResumeRebalanceListener}}) commits every revoked partition. The repository 
> is shared by all the consumers of the endpoint ({{consumersCount}}) and can 
> be shared by several applications.
> # Consumer A processes partition P up to offset 100 and saves it; a rebalance 
> moves P to consumer B.
> # B resumes at 101, processes up to 500 and saves 500.
> # P comes back to A (B leaves, reconnects, scale down; several rebalances in 
> a row are common during a rolling restart): 
> {{OffsetPartitionAssignmentAdapter}} seeks to 501.
> # Another rebalance revokes P before A polled a record of it: the revoke 
> saves A's cached 100. The repository goes back from 500 to 100, and the next 
> owner processes 101..500 again.
> No message is lost, but every record processed since A last owned P is 
> processed again.
> h3. How to reproduce
> {{CommitToOffsetManagerRevokeTest}}: a {{MockConsumer}}, a 
> {{MemoryStateRepository}}, a {{CommitToOffsetManager}} and the 
> {{ClassicRebalanceListener}}: save 100, revoke, set the repository to 500 
> (the other consumer), assign (position 501), revoke: the repository holds 100.
> h3. Fix
> {{CommitToOffsetManager.commit(partition)}} removes the offset from its cache 
> after saving it, as the other commit managers do.
> _Filed with Claude Code on behalf of allthingssecurity._



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to