Tom Thank you for the suggestion. After redeploy and changing to 1 spout I see that topology stops for a few min and starts again. The max tuples is set to 8k and the topology has just a few bolts that write back to Kafka.
Anything else I may do? Sent from my iPhone > On Jul 22, 2016, at 3:06 AM, Ziemer, Tom <[email protected]> wrote: > > Hi, > > it might be the automatic backpressure feature (which is enabled by default > in 1.0.1). Check in your zookeeper whether you find a backpressure entry in > /storm/backpressure. > > If this is the case, you can disable the feature via: > topology.backpressure.enable > > Regards, > Tom > > From: Yaroslav Chinskiy [mailto:[email protected]] > Sent: Freitag, 22. Juli 2016 04:01 > To: [email protected] > Subject: Re: KafkaSpout in 1.0.1 stops processing messages > > In my case deactivate / activate does not work. I have to redeploy. > > > Sent from my iPad > > On Jul 21, 2016, at 9:50 PM, Rahul Chugh <[email protected]> wrote: > > We are facing a similar problem. Randomly the spout stops consuming the > messages from the Kafka topic. A restart of the topology makes it to start > consuming again. > > Any help here is appreciated. > > Thanks > Rahul > > > > > > On Thu, Jul 21, 2016 at 6:14 PM -0700, "Yaroslav Chinskiy" <[email protected]> > wrote: > > Hi, > > > I have an issue where KafkaSpout randomly stops processing messages. I am > using the old implementation not storm-kaka-client > > I use Kafka 0.10 and Storm 1.0.1. I have 3 node cluster with 4 spouts across. > > To test the issue I stream 10M messages to Kafka and consume them with > KafkaSpout. The issue is random and I am not sure where to start > investigation. > Any help to understand the issue and start investigation is highly > appreciated. > > > My experience is similar to what is reported here > http://stackoverflow.com/questions/38198961/storm-kafkaspout-stopped-to-consume-messages-from-kafka-topic > > Thank you > Yarik. >
