shashank created CAMEL-25838:
--------------------------------
Summary: camel-iggy - after one failed poll the consumer polls
with a new client that never joined the consumer group, and consumes nothing
again until the route restarts
Key: CAMEL-25838
URL: https://issues.apache.org/jira/browse/CAMEL-25838
Project: Camel
Issue Type: Bug
Reporter: shashank
Assignee: shashank
The iggy consumer joins its consumer group once: {{IggyConsumer.doStart}}
borrows a client from its pool and calls
{{IggyEndpoint.initializeConsumerGroup}} (which ends with
{{joinConsumerGroup}}), then gives the client back (main 20cdb309f7d0). Since
CAMEL-25289 (4.23.0) {{IggyFetchRecords.pollMessages}} discards the client of a
failed poll ({{invalidateClient}}), so that a broken connection is not reused;
the pool then creates a new client with {{IggyClientFactory.create}}, which
only logs in.
The Iggy server keeps the consumer group membership per connection. Its client
contract ({{MessageClient.poll_messages}} in apache/iggy) says that polling a
consumer group the client is not, or no longer, a member of fails with
{{ConsumerGroupMemberNotFound}} (error 5006) "so the caller can rejoin". Camel
does not rejoin: after one failed poll (a dropped connection, a timeout, a
server restart) every later poll fails with that error, the consumer logs it
once per second, and nothing is consumed until the route is restarted. The
messages stay in the stream, but the route has stopped for good while it looks
started.
*Fix*: the consumer remembers which pooled clients joined the group (the client
used at start joined in {{initializeConsumerGroup}}), makes a client join
before its first poll ({{IggyConsumer.joinConsumerGroupIfNeeded}}, the join
part of {{initializeConsumerGroup}} is now {{IggyEndpoint.joinConsumerGroup}}),
and forgets a client when it is discarded. A join that fails counts as a failed
poll: the client is discarded and the next poll tries a new one.
*Test*: new {{IggyConsumerRejoinGroupTest}} with mocked Iggy clients that
behave like the server here: the first client fails its poll ("Connection
reset"); a client that did not join gets {{IggyResourceNotFoundException}} with
{{CONSUMER_GROUP_MEMBER_NOT_FOUND}}, a member gets one message. On main it
fails ({{expected: <[after the failed poll]> but was: <[]> within 20
seconds}}); with the fix it passes. camel-iggy unit tests with the fix: 18 run,
0 failures (the integration tests need a container and were not run).
*How it was found*: variant analysis of CAMEL-25815 (pattern: a consumer relies
on server-side state that belongs to a connection or session, and the reconnect
path does not create it again), then the JUnit test. The server rule is taken
from the Iggy client contract quoted above; it was not run against an Iggy
server.
*Affected*: main and camel-4.23.x (4.23.0), since CAMEL-25289. camel-4.22.x and
older keep the client of a failed poll (the pool then runs out after 8
failures, fixed by CAMEL-25289) and are not affected by this issue (read, not
run).
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
