Thanks for that Mike, I wasn't aware of that, but I'll try that next time.
For completeness, I currently accomplish this by querying a Thrift
connection to
Ninmbus and aggregating counters in the response data structures along
the way
(and then -- side note -- I send the results to Graphite... well,
actually I send
the deltas between two consecutive query results divided by the time in
between
those samples, which is derived rate metric; but one can easily not
divide and
use the running totals for the aggregate). (Which, btw, I use the Python
'collections.Counter' dict module for because there are actually many useful
metrics that come back to keep track of).
That approach is, of course, out-of-band and asynchronous relative to what
the below in-topology java approach provides, but it does work to
give you running totals across the board (acks, emits, transferred, etc.).
Regards,
Noel
Dimension Data, LLC
On 03/05/2014 03:27 PM, Michael Rose wrote:
https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/backtype/storm/hooks/BaseTaskHook.java
Override this guy, apply the task hook in either your Storm Conf or in
the bolt its self.
List<String> listOfHooks = Lists.newArrayList(MyTaskHook.class.getName());
stormConfig.put(Config.TOPOLOGY_AUTO_TASK_HOOKS, listOfHooks);
OR
In prepare(), topologyContext.addTaskTook()
Michael Rose (@Xorlev <https://twitter.com/xorlev>)
Senior Platform Engineer, FullContact <http://www.fullcontact.com/>
[email protected] <mailto:[email protected]>
On Wed, Mar 5, 2014 at 1:12 PM, Brian O'Neill <[email protected]
<mailto:[email protected]>> wrote:
Sorry, we may have missed something obvious, but is there a way to
count ack'd tuples? We are looking for a hook that would give us
access to tuple that has been fully processed.
I'm guessing we could override the Spout implementation, but that
seems a bit messy.
We could make it the last thing in the topology, but if tuples
were filtered along the way, we wouldn't see them.
Has anyone done something similar?
-brian
--
Brian ONeill
CTO, Health Market Science (http://healthmarketscience.com)
mobile:215.588.6024 <tel:215.588.6024>
blog: http://brianoneill.blogspot.com/
twitter: @boneill42
