[
https://issues.apache.org/jira/browse/CAMEL-25838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-25838 started by shashank.
----------------------------------------
> camel-iggy - after one failed poll the consumer polls with a new client that
> never joined the consumer group, and consumes nothing again until the route
> restarts
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25838
> URL: https://issues.apache.org/jira/browse/CAMEL-25838
> Project: Camel
> Issue Type: Bug
> Reporter: shashank
> Assignee: shashank
> Priority: Major
>
> The iggy consumer joins its consumer group once: {{IggyConsumer.doStart}}
> borrows a client from its pool and calls
> {{IggyEndpoint.initializeConsumerGroup}} (which ends with
> {{joinConsumerGroup}}), then gives the client back (main 20cdb309f7d0). Since
> CAMEL-25289 (4.23.0) {{IggyFetchRecords.pollMessages}} discards the client of
> a failed poll ({{invalidateClient}}), so that a broken connection is not
> reused; the pool then creates a new client with {{IggyClientFactory.create}},
> which only logs in.
> The Iggy server keeps the consumer group membership per connection. Its
> client contract ({{MessageClient.poll_messages}} in apache/iggy) says that
> polling a consumer group the client is not, or no longer, a member of fails
> with {{ConsumerGroupMemberNotFound}} (error 5006) "so the caller can rejoin".
> Camel does not rejoin: after one failed poll (a dropped connection, a
> timeout, a server restart) every later poll fails with that error, the
> consumer logs it once per second, and nothing is consumed until the route is
> restarted. The messages stay in the stream, but the route has stopped for
> good while it looks started.
> *Fix*: the consumer remembers which pooled clients joined the group (the
> client used at start joined in {{initializeConsumerGroup}}), makes a client
> join before its first poll ({{IggyConsumer.joinConsumerGroupIfNeeded}}, the
> join part of {{initializeConsumerGroup}} is now
> {{IggyEndpoint.joinConsumerGroup}}), and forgets a client when it is
> discarded. A join that fails counts as a failed poll: the client is discarded
> and the next poll tries a new one.
> *Test*: new {{IggyConsumerRejoinGroupTest}} with mocked Iggy clients that
> behave like the server here: the first client fails its poll ("Connection
> reset"); a client that did not join gets {{IggyResourceNotFoundException}}
> with {{CONSUMER_GROUP_MEMBER_NOT_FOUND}}, a member gets one message. On main
> it fails ({{expected: <[after the failed poll]> but was: <[]> within 20
> seconds}}); with the fix it passes. camel-iggy unit tests with the fix: 18
> run, 0 failures (the integration tests need a container and were not run).
> *How it was found*: variant analysis of CAMEL-25815 (pattern: a consumer
> relies on server-side state that belongs to a connection or session, and the
> reconnect path does not create it again), then the JUnit test. The server
> rule is taken from the Iggy client contract quoted above; it was not run
> against an Iggy server.
> *Affected*: main and camel-4.23.x (4.23.0), since CAMEL-25289. camel-4.22.x
> and older keep the client of a failed poll (the pool then runs out after 8
> failures, fixed by CAMEL-25289) and are not affected by this issue (read, not
> run).
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
