Hi Dev team, Could you Please let me know if any updates on my query. Thanks, Santhosh Aditya
From: Marigowda, Santhosh Aditya Sent: Wednesday, January 31, 2024 8:50 AM To: dev@kafka.apache.org Cc: Jain, Ankit <ankit.j...@unisys.com>; Namboodiri, Vishnu <vishnu.nambood...@in.unisys.com>; Mudlapur, Rajesh <rajesh.mudla...@au.unisys.com>; Reddy, Thanmai <thanmai.re...@unisys.com> Subject: RE: Kafka consumer group crashing and not able to consume once service is up Thanks Matthias for your response. Sorry, my bad please find the updated Image. · We have 1 topic and 4 partitions, Each consumer is pointing to the topic. There is no partition assignment specific to the consumer group from our software code. · Please let me know if you need Kafka server logs or consumer service logs, Post restart of our service no logs are printed in onMessage method. · We are using consumer.subscribe(topic) [cid:image001.png@01DA5A80.0A8A7210] Thanks, Santhosh Aditya From: Matthias J. Sax <mj...@apache.org<mailto:mj...@apache.org>> Sent: Wednesday, January 31, 2024 12:13 AM To: dev@kafka.apache.org<mailto:dev@kafka.apache.org> Subject: Re: Kafka consumer group crashing and not able to consume once service is up I am a not sure if I can follow completely. From the figures you show, you have a topic with 4 partitions, and 4 consumer groups. Thus, each consumer group should read all 4 partitions, but the figure indicate that each group would read a single I am a not sure if I can follow completely. From the figures you show, you have a topic with 4 partitions, and 4 consumer groups. Thus, each consumer group should read all 4 partitions, but the figure indicate that each group would read a single partition only? Can you clarify? Are you using `consumer.subscribe` or `consumer.assign`? In general, might be good too collect some INFO (or DEBUG) level logs for the crashing service after restart to see what it's doing. -Matthias On 1/30/24 7:17 AM, Marigowda, Santhosh Aditya wrote: > Hi Kafka Dev Team, > > Could you please help us with our problem. > > In our POC, if one of the kafka consumer(Service) shuts down or crashes > then post restart of service none of the messages are getting consumed > by the crashed Service. > > Other services are consuming without any issues. > > One of service crash/Shutdown > > If we rename the Kafka consumer group name and start the service then > messages start consuming. > > Consumer configuration : > > { > > delay:1000 > > timeout:0 > > topic-name= test > > handlers=["Listener"] > > source="kafka-consumer" > > enable_auto_commit="false" > > group="Consumer-Group-1" > > } > > local-kafka-consumer = { > > server= > "{kafka-hostname}:{kafka-port}" > > > deserializer="org.apache.kafka.common.serialization.StringDeserializer" > > auto_offset_reset="latest" > > enable_auto_commit="true" > > maxRequestSize=20971520 > > } > > Thanks, > > Santhosh Aditya >
