Hi Shantanu,
If you are using kafka stream, upgrade to the latest jar. There are a bunch
of fixes in the way it uses kafka consumers.
Apart from this: try these settings
1. Set the session.timeout.ms value higher, to something like 30
2. Set the heartbeat.interval.ms to lower value, something
Hey Vincent.
That's exactly how my code is. I am doing processing within that for loop.
In KIP-62 I read that heartbeat happens via a separate thread
https://github.com/dpkp/kafka-python/issues/948. But you are saying it
happens through polling. What can be considered true? I have set
session.tim
Shantanu, I was more referering to you application code.
You should have something similar to :
while (true) {
ConsumerRecords records = consumer.poll(100);
for (ConsumerRecord record : records) {
// Your logic
}
}
You should make sure that the code within the loop doesn't t
Another observation is that when I restart my application. Consumption
doesn't start till 5-6 minutes. In kafka consumer logs I see
ConsumerCoordinator.333 - Revoking previously assigned partitions [] for
group notifications-consumer
AbstractCoordinator:381 - (Re-)joining group notifications-consu
Hi Vincent,
Yes I reduced max.poll.records to get that same effect. I reduced it all
the way down to 5 records still I am seeing same error. What else can be
done? For one topic I can see that a single message processing is taking
about 20 seconds. So 5 of them will take 1 minute. So I set
session
Hi M. Manna,
Thanks I will try these settings.
On Thu, May 24, 2018 at 5:15 PM M. Manna wrote:
> Set your rebalance.backoff.ms=1 and zookeeper.session.timeout.ms=3
> in addition to what Manikumar said.
>
>
>
> On 24 May 2018 at 12:41, Shantanu Deshmukh wrote:
>
> > Hello,
> >
> > There
Hello Shantanu,
It is also important to consider your consumer code. You should not spend
to much time in between two calls to "poll" method. Otherwise, the consumer
not calling poll will be considered dead by the group, triggering a
rebalancing.
Best
On Thu, May 24, 2018 at 1:45 PM M. Manna wr
Set your rebalance.backoff.ms=1 and zookeeper.session.timeout.ms=3
in addition to what Manikumar said.
On 24 May 2018 at 12:41, Shantanu Deshmukh wrote:
> Hello,
>
> There was a type in my first mail. session.timeout.ms is actually 6
> not
> 6000. So it is less than heartbeat.inter
Hello,
There was a type in my first mail. session.timeout.ms is actually 6 not
6000. So it is less than heartbeat.interval.ms.
On Thu, May 24, 2018 at 2:46 PM Manikumar wrote:
> heartbeat.interval.ms should be lower than session.timeout.ms.
>
> Check here:
> http://kafka.apache.org/0101/doc
heartbeat.interval.ms should be lower than session.timeout.ms.
Check here:
http://kafka.apache.org/0101/documentation.html#newconsumerconfigs
On Thu, May 24, 2018 at 2:39 PM, Shantanu Deshmukh
wrote:
> Someone please help me. I am suffering due to this issue since a long time
> and not finding
Someone please help me. I am suffering due to this issue since a long time
and not finding any solution.
On Wed, May 23, 2018 at 3:48 PM Shantanu Deshmukh
wrote:
> We have a 3 broker Kafka 0.10.0.1 cluster. There we have 3 topics with 10
> partitions each. We have an application which spawns thr
We have a 3 broker Kafka 0.10.0.1 cluster. There we have 3 topics with 10
partitions each. We have an application which spawns threads as consumers.
We spawn 5 consumers for each topic. I am observing that consider group
randomly keeps rebalancing. Then many times we see logs saying "Revoking
part
12 matches
Mail list logo
