Hi Santhosh,
Your problem statement confuses me a bit (apologize). You mentioned "if one
of the kafka consumer(Service)" - Do you have a single member consumer
group? Could you elaborate on the setup a bit? Did you also mean after
restarting the "service", the service was not able to resume from consuming
the messages?
Logs would be very helpful to see why these consumers stopped consuming
messages. If you include the logs (that would be really helpful), could you
also include the client version.
P
On Wed, Feb 7, 2024 at 9:45 PM Marigowda, Santhosh Aditya
<santhosh.marigo...@unisys.com.invalid> wrote:
> Hi Dev team,
>
> Could you Please let me know if any updates on my query.
>
>
>
> Thanks,
>
> Santhosh Aditya
>
>
>
> *From:* Marigowda, Santhosh Aditya
> *Sent:* Wednesday, January 31, 2024 8:50 AM
> *To:* dev@kafka.apache.org
> *Cc:* Jain, Ankit <ankit.j...@unisys.com>; Namboodiri, Vishnu <
> vishnu.nambood...@in.unisys.com>; Mudlapur, Rajesh <
> rajesh.mudla...@au.unisys.com>; Reddy, Thanmai <thanmai.re...@unisys.com>
> *Subject:* RE: Kafka consumer group crashing and not able to consume once
> service is up
>
>
>
> Thanks Matthias for your response. Sorry, my bad please find the updated
> Image.
>
> · We have 1 topic and 4 partitions, Each consumer is pointing to the
> topic. There is no partition assignment specific to the consumer group from
> our software code.
>
> · Please let me know if you need Kafka server logs or consumer service
> logs, Post restart of our service no logs are printed in onMessage method.
>
> · We are using consumer.subscribe(topic)
>
>
>
>
>
>
>
> Thanks,
>
> Santhosh Aditya
>
>
>
> *From:* Matthias J. Sax <mj...@apache.org>
> *Sent:* Wednesday, January 31, 2024 12:13 AM
> *To:* dev@kafka.apache.org
> *Subject:* Re: Kafka consumer group crashing and not able to consume once
> service is up
>
>
>
> I am a not sure if I can follow completely. From the figures you show, you
> have a topic with 4 partitions, and 4 consumer groups. Thus, each consumer
> group should read all 4 partitions, but the figure indicate that each group
> would read a single
>
> I am a not sure if I can follow completely.
>
>
>
> From the figures you show, you have a topic with 4 partitions, and 4
>
> consumer groups. Thus, each consumer group should read all 4 partitions,
>
> but the figure indicate that each group would read a single partition only?
>
>
>
> Can you clarify? Are you using `consumer.subscribe` or `consumer.assign`?
>
>
>
> In general, might be good too collect some INFO (or DEBUG) level logs
>
> for the crashing service after restart to see what it's doing.
>
>
>
>
>
> -Matthias
>
>
>
> On 1/30/24 7:17 AM, Marigowda, Santhosh Aditya wrote:
>
> > Hi Kafka Dev Team,
>
> >
>
> > Could you please help us with our problem.
>
> >
>
> > In our POC, if one of the kafka consumer(Service) shuts down or crashes
>
> > then post restart of service none of the messages are getting consumed
>
> > by the crashed Service.
>
> >
>
> > Other services are consuming without any issues.
>
> >
>
> > One of service crash/Shutdown
>
> >
>
> > If we rename the Kafka consumer group name and start the service then
>
> > messages start consuming.
>
> >
>
> > Consumer configuration :
>
> >
>
> > {
>
> >
>
> > delay:1000
>
> >
>
> > timeout:0
>
> >
>
> > topic-name= test
>
> >
>
> > handlers=["Listener"]
>
> >
>
> > source="kafka-consumer"
>
> >
>
> > enable_auto_commit="false"
>
> >
>
> > group="Consumer-Group-1"
>
> >
>
> > }
>
> >
>
> > local-kafka-consumer = {
>
> >
>
> > server=
>
> > "{kafka-hostname}:{kafka-port}"
>
> >
>
> >
>
> > deserializer="org.apache.kafka.common.serialization.StringDeserializer"
>
> >
>
> > auto_offset_reset="latest"
>
> >
>
> > enable_auto_commit="true"
>
> >
>
> > maxRequestSize=20971520
>
> >
>
> > }
>
> >
>
> > Thanks,
>
> >
>
> > Santhosh Aditya
>
> >
>
>
