Another approach is to add a ITaskHook implementation to understand the actual latency in your bolt for every ack..An example using ITaskHook can be found at http://iamparv.blogspot.com/2013/06/storm-itaskhook-implementation.html
Using the BoltAckInfo, we can find the actuals. From: Seungtack Baek [mailto:[email protected]] Sent: Monday, June 22, 2015 2:31 PM To: [email protected] Subject: Re: How fast can bolt access emitted data You might want to check your parallelism. Are all those "10" messages in bolt B,C,etc are being processed in parallel? How many workers are you using? How many cores do you have for each node? [Image removed by sender.] Seungtack Baek | Precocity, LLC Tel/Direct: (972) 378-1030 | Mobile: (214) 477-5715 [email protected]<mailto:[email protected]> | www.precocityllc.com<http://www.precocityllc.com/> This is the end of this message. -- On Mon, Jun 22, 2015 at 11:30 AM, Kashyap Mhaisekar <[email protected]<mailto:[email protected]>> wrote: Has no one faced this issue? Thanks Kashyap On Jun 21, 2015 14:57, "Kashyap Mhaisekar" <[email protected]<mailto:[email protected]>> wrote: Hi, I have a topology like the below - A (Spout) ->(Emits a number say 1000) -> B (bolt) [Receives this number and divides this into 10 emits of 100 each) -> C (bolt) [Recieves these emits and divides this to 10 emits of 10 numbers) -> D (bolt) [Does some computation on the number and emits one message] -> E (bolt) [Aggregates all the data and confirms if all the 1000 messages are processed) Every bolt takes under 3 msec to complete and as a result, I estimated that the end to end processing for 1000 takes not more than 50 msec including any latencies. But what I noticed is that its takes close to 200 msec and the Emit from a spout/bolt and getting it to bolt takes close to 6 ms on a an average. I am not able to figure out why it takes so much time to come between a spout/bolt to next bolt. I understand that the spout/bolt buffers the data into a queue and then the subsequent bolt consumes from there. Any pointers on this will help. Thanks Kashyap This email and any files transmitted with it are confidential, proprietary and intended solely for the individual or entity to whom they are addressed. If you have received this email in error please delete it immediately.
