Federico Mariani created CAMEL-27022:
----------------------------------------

             Summary: camel-kafka - with consumersCount greater than 1, every 
consumer gets the same groupInstanceId and they fence each other
                 Key: CAMEL-27022
                 URL: https://issues.apache.org/jira/browse/CAMEL-27022
             Project: Camel
          Issue Type: Bug
          Components: camel-kafka
            Reporter: Federico Mariani


With {{groupInstanceId}} set and {{consumersCount}} greater than 1, every 
consumer thread of the endpoint gets the same {{group.instance.id}}. A static 
member id is allowed once in a consumer group, so the consumers fence each 
other from the start: the join and the polls fail with 
{{FencedInstanceIdException}} ("another consumer with the same 
group.instance.id has registered with a different member.id"). With the default 
{{pollOnError=ERROR_HANDLER}} this repeats every few milliseconds, each time 
with a WARN and an ERROR from the error handler. The group never has more than 
one member. The same happens with {{additionalProperties.group.instance.id}}. 
The consumers also share the {{client.id}} that the Kafka client derives from 
the instance id ({{consumer-<group>-<instance id>}}).

{code:java}
// topic with 3 partitions
from("kafka:orders?groupId=orders&groupInstanceId=pod-1&consumersCount=3")
    .to("log:orders"); // FencedInstanceIdException in a loop, one group member 
at most
{code}

Expected: each consumer is its own static member. With {{consumersCount}} 
greater than 1, the consumer with index n gets {{<groupInstanceId>-n}} 
({{pod-1-0}}, {{pod-1-1}}, {{pod-1-2}}), as Kafka Streams does for its stream 
threads. The derived ids also give each consumer its own client id. With 
{{consumersCount=1}} the id stays as configured, so an existing static member 
keeps its identity.

_Claude Code on behalf of Federico Mariani_



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to