sub millisecond latency with storm

In my topology I see around 1 - 2 ms latency when transferring tuples from spouts to bolts or from bolts to bolts. I am calculating latency using nanosecond timestamps because the whole topology runs inside a single worker. Topology is run in a cluster which runs in a production capable

RE: is stateful bolts production ready?

Hi Arun, Could you please help me with my questions 2 and 3 if possible? Thanks Manusha From: Arun Iyer [mailto:ai...@hortonworks.com] On Behalf Of Arun Mahadevan Sent: 11 August 2017 10:20 To: Wijekoon, Manusha [ICG-IT] Cc: user@storm.apache.org Subject: Re: is stateful bolts production ready

RE: is stateful bolts production ready?

=DwMFaQ=j-EkbjBYwkAB4f8ZbVn1Fw=3V6DSqhjAEmq5iy51r9vVgFw9iAHiTSNsZl3DKb4ONM=afgts--lg7Jf3oTEhOyGvkwmkT8RVx1LedYRwfuTwLg=dV6qlBomTiYIN23BV3fzJl7nhJBd9ewoFsDi3HkxD6I=> Thanks, Arun From: "Wijekoon, Manusha" <manusha.wijek...@citi.com<mailto:manusha.wijek...@citi.com>> Reply-To:

is stateful bolts production ready?

Hello I am thinking of using stateful bolts to manage state of a bolt. From the documentation it is not clear how to save the bolt state however. I understand it has to be done when we process the checkpoint tuple, but how? Do I just need to update the state object and storm pick it up during

Meauring latency of Storm bolts

I am trying to measure latency of each bolt in a topology. The latency numbers given by Storm is not enough as we want to calculate percentiles. In my current setup, I measure latency of the bolt by measuring time it takes to complete the execute method including call to emit. The assumption

Support of topic wildcards in Kafka spout

We have a topology that has multiple kafka spout tasks. Each spout task is supposed to read a subset of messages from a set of Kafka topics. Topics have to be subscribed using a wild card such as AAA.BBB.*. The expected behaviour would be that all spout tasks collectively will consume all