Federico Mariani created CAMEL-27022:
----------------------------------------
Summary: camel-kafka - with consumersCount greater than 1, every
consumer gets the same groupInstanceId and they fence each other
Key: CAMEL-27022
URL: https://issues.apache.org/jira/browse/CAMEL-27022
Project: Camel
Issue Type: Bug
Components: camel-kafka
Reporter: Federico Mariani
With {{groupInstanceId}} set and {{consumersCount}} greater than 1, every
consumer thread of the endpoint gets the same {{group.instance.id}}. A static
member id is allowed once in a consumer group, so the consumers fence each
other from the start: the join and the polls fail with
{{FencedInstanceIdException}} ("another consumer with the same
group.instance.id has registered with a different member.id"). With the default
{{pollOnError=ERROR_HANDLER}} this repeats every few milliseconds, each time
with a WARN and an ERROR from the error handler. The group never has more than
one member. The same happens with {{additionalProperties.group.instance.id}}.
The consumers also share the {{client.id}} that the Kafka client derives from
the instance id ({{consumer-<group>-<instance id>}}).
{code:java}
// topic with 3 partitions
from("kafka:orders?groupId=orders&groupInstanceId=pod-1&consumersCount=3")
.to("log:orders"); // FencedInstanceIdException in a loop, one group member
at most
{code}
Expected: each consumer is its own static member. With {{consumersCount}}
greater than 1, the consumer with index n gets {{<groupInstanceId>-n}}
({{pod-1-0}}, {{pod-1-1}}, {{pod-1-2}}), as Kafka Streams does for its stream
threads. The derived ids also give each consumer its own client id. With
{{consumersCount=1}} the id stays as configured, so an existing static member
keeps its identity.
_Claude Code on behalf of Federico Mariani_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)