Michael,
That's good to know. I was unaware. That said, if execution of a bolt has not
occurred, I would still expect a 0 emit count and acks not to be falling behind
the emits by much. My acks are half my emits.
From: Michael Rose <[email protected]>
To: "[email protected]" <[email protected]>; Jason Kania
<[email protected]>
Sent: Monday, February 23, 2015 3:52 PM
Subject: Re: Question About Emitted, Transferred and Acked Bolts
Keep in mind that those metrics are sampled at the rate of
topology.stats.sample.rate, 0.05 by default. If you turn it up to 1.0 you'll
see full-resolution, though at the price of more time spent collecting metrics.
Michael RoseSenior Software EngineerFullContact | fullcontact.comm:
+1.720.837.1357 | t: @xorlev
All Your Contacts, Updated and In One Place.Try FullContact for Free
On Mon, Feb 23, 2015 at 12:14 PM, Jason Kania <[email protected]> wrote:
I have two comments to add:
1) Is there any JIRA for invalid metrics values? I did not see one. I am
running with bolts having breakpoints and long before my bolts are every
entered, the metrics indicate that these bolts already have more than 100
emits. I have thought to raise a JIRA on this but I am not sure what I would
add for details. Would some specific debug output aid in resolving this?
2) For acks, is there any possibility of adding tracking for acks that happen
after a timeout? I can step into my bolt each time it is called and confirm
that it is acking each request, yet the acks do not match the emits (which
should have a 1 to 1 ratio). I am guessing that this is because the ack
happened too late or it might be incorrect metrics total.
I use the STORM UI for processing tracking.
Thanks,
Jason
From: Nathan Leung <[email protected]>
To: user <[email protected]>
Sent: Monday, February 23, 2015 11:56 AM
Subject: Re: Question About Emitted, Transferred and Acked Bolts
executed = # of times you called executedacked = # of executed tuples that you
acked; ideally this will match executedemitted = # of tuples that you emitted;
if you call emit more than once per execute call this can be higher than
execute counttransferred = # of tuples transferred downstream; if you have 2
bolts subscribing to your bolt, then this count can be higher than emitted.
On Mon, Feb 23, 2015 at 11:35 AM, Rahul Reddy <[email protected]> wrote:
Hi,
Can you guys help me understand difference between emitted, transferred and
acked tuples.
In my case every tuple emitted by ablog-filter-bolt will be processed by
ablog-flatten-xml-bolt which will then be written by ablog-hdfs-bolt to hdfs.
Ideally all metrics for executed/acked should match after tuples are emitted
from ablog-filter-bolt . I'm not sure why there is so much discrepancy in
emitted/transferredacked tuple count between these bolts although it dosent
show any failed tuples.
Any ideas what I can check and how to interpret metrics correctly?
Thanks
Rahul
