Re: Introduce lag into a topology spout to process the events with a delay.

Yes. I am reading both the streams from kafka as part of a topology. On Wed, Feb 22, 2017 at 12:39 AM, Ankur Garg wrote: > Hi Sandeep , > > One question :- how are you reading Streams B and A . Are u reading from > some messaging queue (Kafka , Rabbit Mq etc.) with some

Re: Storm Kafka offset monitoring

Hi Priyank Currently I tested the spout using zookeeper broker hosts .This is processing fine.But I saw another way of initialising spout using Kafka bootstrap severs using kafkaspoutconfig class.

Re: Storm Kafka offset monitoring

Hi Pradeep, A release vote for RC1 of 1.1.0 is in progress. You can track that and once it gets released you can upgrade. Regarding upgrading your spout, you don’t need to make any code changes. You just need to use the latest released spout code. Usually it involves updating a pom file that

Re: Storm Kafka offset monitoring

Hi Priyank Thanks for your reply.i was not able to find 1.1.0 version of storm Can you please point to that.also can you please confirm on what specific spout changes to make. Regards Pradeep S On Tue, Feb 21, 2017 at 10:54 AM Priyank Shah wrote: > Hi Pradeep, > > > > If

Re: Kafka Spout enable.auto.commit=false

Thanks, Hugo. That's all I want to know abot this. On Tue, Feb 21, 2017 at 9:01 PM, Hugo Da Cruz Louro wrote: > As per KafkaConsumer documentation > > : > > “ Setting

Re: Introduce lag into a topology spout to process the events with a delay.

Hi Sandeep , One question :- how are you reading Streams B and A . Are u reading from some messaging queue (Kafka , Rabbit Mq etc.) with some spout (as part of some topology) reading from them . Please confirm . Thanks Ankur On Tue, 21 Feb 2017 at 15:28 Sandeep Samudrala

Re: Storm Kafka offset monitoring

Hi Pradeep, If you upgrade your spout in the topology and storm code to a later version(I checked v1.1.0 and it has the tool) you will get a table in storm ui which show you offsets. If you cannot upgrade then I think you will have to do it manually. From: pradeep s

Re: Kafka Spout enable.auto.commit=false

As per KafkaConsumer documentation: “ Setting enable.auto.commit=true means that offsets are committed automatically with a frequency controlled by the config auto.commit.interval.ms. “ That

Re: Kafka spout stops commiting offsets on some partitions

Hi Igor, In normal operation this shouldn’t happen, it it could have happened due to a consumer rebalance. We have improved the code recently. From this info alone it’s hard to tell what really happened. I could suggest a couple of things. 1. Try reproducing the issue again 2. Enable logs to

Storm Kafka offset monitoring

Hi , I am using Storm 1.0.2 and Kafka 0.10.1.1 versions . Storm spout is configured using zookeeper broker hosts. Is there a monitoring ui which i can use to track the consumer offsets and lag. I was using yahoo kafka manager , but its showing storm spout as a consumer.Any help? Regards Pradeep S

RE: Local Mode issues and undocumented behaviour

HI Petr, I encountered a similar issue to your first point a while ago. See https://issues.apache.org/jira/browse/STORM-2038 (STORM-2038) for some more discussion that occurred. Regards, Paul -Original Message- From: Petr Janeček [mailto:janecekp...@seznam.cz] Sent: 20 February 2017

Introduce lag into a topology spout to process the events with a delay.

Hello, I have two streams A and B. I need to enrich events coming from stream B with events coming from A and I store events coming from A in a key-value store to enrich events from B. Events that doesn't get enriched are sent to a deferred queue(kafka stream) and are read back later. Most of