Nathan,
I only get spout failures. None of my bolts fail. Are all failures on a spout
going to be associated with timeout? If not, how would one know the difference?
Hence my question about whether it would be possible to track timed-out acks
explicitly.
Thanks,
Jason
From: Nathan Leung <[email protected]>
To: user <[email protected]>; Jason Kania <[email protected]>
Sent: Monday, February 23, 2015 4:52 PM
Subject: Re: Question About Emitted, Transferred and Acked Bolts
If your processing takes a long time they may be timing out.
On Mon, Feb 23, 2015 at 4:35 PM, Jason Kania <[email protected]> wrote:
Thanks for the suggestion. My traffic is only one or two tuples per second in
the current environment so I would not expect that to be the problem. I do have
failures. Hence, I was wondering if slow acking was the problem. In stepping
through the processing, I know the failures aren't in the bolts themselves.
That is why I thought it strange.
From: Michael Rose <[email protected]>
To: "[email protected]" <[email protected]>; Jason Kania
<[email protected]>
Sent: Monday, February 23, 2015 4:24 PM
Subject: Re: Question About Emitted, Transferred and Acked Bolts
Do you have enough ackers to keep up with your traffic? How about failures?
Michael RoseSenior Software EngineerFullContact | fullcontact.comm:
+1.720.837.1357 | t: @xorlev
All Your Contacts, Updated and In One Place.Try FullContact for Free
On Mon, Feb 23, 2015 at 2:16 PM, Jason Kania <[email protected]> wrote:
Michael,
That's good to know. I was unaware. That said, if execution of a bolt has not
occurred, I would still expect a 0 emit count and acks not to be falling behind
the emits by much. My acks are half my emits.
From: Michael Rose <[email protected]>
To: "[email protected]" <[email protected]>; Jason Kania
<[email protected]>
Sent: Monday, February 23, 2015 3:52 PM
Subject: Re: Question About Emitted, Transferred and Acked Bolts
Keep in mind that those metrics are sampled at the rate of
topology.stats.sample.rate, 0.05 by default. If you turn it up to 1.0 you'll
see full-resolution, though at the price of more time spent collecting metrics.
Michael RoseSenior Software EngineerFullContact | fullcontact.comm:
+1.720.837.1357 | t: @xorlev
All Your Contacts, Updated and In One Place.Try FullContact for Free
On Mon, Feb 23, 2015 at 12:14 PM, Jason Kania <[email protected]> wrote:
I have two comments to add:
1) Is there any JIRA for invalid metrics values? I did not see one. I am
running with bolts having breakpoints and long before my bolts are every
entered, the metrics indicate that these bolts already have more than 100
emits. I have thought to raise a JIRA on this but I am not sure what I would
add for details. Would some specific debug output aid in resolving this?
2) For acks, is there any possibility of adding tracking for acks that happen
after a timeout? I can step into my bolt each time it is called and confirm
that it is acking each request, yet the acks do not match the emits (which
should have a 1 to 1 ratio). I am guessing that this is because the ack
happened too late or it might be incorrect metrics total.
I use the STORM UI for processing tracking.
Thanks,
Jason
From: Nathan Leung <[email protected]>
To: user <[email protected]>
Sent: Monday, February 23, 2015 11:56 AM
Subject: Re: Question About Emitted, Transferred and Acked Bolts
executed = # of times you called executedacked = # of executed tuples that you
acked; ideally this will match executedemitted = # of tuples that you emitted;
if you call emit more than once per execute call this can be higher than
execute counttransferred = # of tuples transferred downstream; if you have 2
bolts subscribing to your bolt, then this count can be higher than emitted.
On Mon, Feb 23, 2015 at 11:35 AM, Rahul Reddy <[email protected]> wrote:
Hi,
Can you guys help me understand difference between emitted, transferred and
acked tuples.
In my case every tuple emitted by ablog-filter-bolt will be processed by
ablog-flatten-xml-bolt which will then be written by ablog-hdfs-bolt to hdfs.
Ideally all metrics for executed/acked should match after tuples are emitted
from ablog-filter-bolt . I'm not sure why there is so much discrepancy in
emitted/transferredacked tuple count between these bolts although it dosent
show any failed tuples.
Any ideas what I can check and how to interpret metrics correctly?
Thanks
Rahul
