hmm I haven't checked the latency between bolts.. but do you think 15ms could've come from network?
*Seungtack Baek | Precocity, LLC* Tel/Direct: (972) 378-1030 | Mobile: (214) 477-5715 *[email protected] <[email protected]>* | www.precocityllc.com This is the end of this message. -- On Tue, Jun 23, 2015 at 4:01 AM, Kashyap Mhaisekar <[email protected]> wrote: > It is a 5 machines os 4 core 8 gb. > Parallelism is set to 100. Whether I have 1000 messages or 100, I see > greeter than 10-15ms from the time bolt emits and the next bolt consumes... > :( > > Vidya, thanks. Will check the ITaskHook. > > What is the typical latency you guys see? > > Thanks > Kashyap > On Jun 22, 2015 22:50, "Srividhya Shanmugam" <[email protected]> > wrote: > >> Another approach is to add a ITaskHook implementation to understand the >> actual latency in your bolt for every ack..An example using ITaskHook can >> be found at >> http://iamparv.blogspot.com/2013/06/storm-itaskhook-implementation.html >> >> >> >> Using the BoltAckInfo, we can find the actuals. >> >> >> >> *From:* Seungtack Baek [mailto:[email protected]] >> *Sent:* Monday, June 22, 2015 2:31 PM >> *To:* [email protected] >> *Subject:* Re: How fast can bolt access emitted data >> >> >> >> You might want to check your parallelism. Are all those "10" messages in >> bolt B,C,etc are being processed in parallel? How many workers are you >> using? How many cores do you have for each node? >> >> >> [image: Image removed by sender.] >> >> *Seungtack Baek | Precocity, LLC* >> >> Tel/Direct: (972) 378-1030 | Mobile: (214) 477-5715 >> >> *[email protected] <[email protected]>* | >> www.precocityllc.com >> >> >> >> This is the end of this message. >> >> -- >> >> >> >> On Mon, Jun 22, 2015 at 11:30 AM, Kashyap Mhaisekar <[email protected]> >> wrote: >> >> Has no one faced this issue? >> >> Thanks >> Kashyap >> >> On Jun 21, 2015 14:57, "Kashyap Mhaisekar" <[email protected]> wrote: >> >> Hi, >> >> I have a topology like the below - >> >> A (Spout) ->(Emits a number say 1000) -> B (bolt) [Receives this number >> and divides this into 10 emits of 100 each) -> C (bolt) [Recieves these >> emits and divides this to 10 emits of 10 numbers) -> D (bolt) [Does some >> computation on the number and emits one message] -> E (bolt) [Aggregates >> all the data and confirms if all the 1000 messages are processed) >> >> >> >> Every bolt takes under 3 msec to complete and as a result, I estimated >> that the end to end processing for 1000 takes not more than 50 msec >> including any latencies. But what I noticed is that its takes close to 200 >> msec and the Emit from a spout/bolt and getting it to bolt takes close to 6 >> ms on a an average. I am not able to figure out why it takes so much time >> to come between a spout/bolt to next bolt. I understand that the spout/bolt >> buffers the data into a queue and then the subsequent bolt consumes from >> there. >> >> >> >> Any pointers on this will help. >> >> >> >> Thanks >> >> Kashyap >> >> >> >> This email and any files transmitted with it are confidential, >> proprietary and intended solely for the individual or entity to whom they >> are addressed. If you have received this email in error please delete it >> immediately. >> >
