I used Nimbus Thrift on my end.
The one thing to note about that approach is that, if you're planning to
plot
those metrics (say, on Graphite), the results aren't very interesting
because the
metrics -- emitted, transferred, acks, etc. -- are aggregates and simply
monotonically
increase, which lead to not so insightful graphs (... more-or-less lines
that
go up from left-to-right forever).
However, I took consecutive metrics samples from Nimbus, say,
every 5 seconds, and plotted the difference between the two samples
divided by 5 to plot
average quantities instead. That was insightful. So while (except for
latency
type metrics) Nimbus doesn't provide you with average rates for
emits/transferred/acks
(and you won't see them on the UI either), you can easily derive them
manually
and send that to Graphite (or to your charting platform). If using
Graphite, btw.,
you also have the option of letting Graphite calculate the averages on
the back-end
for you, too (giving you options).
Finally, for application specific metrics exposed via Java Mbeans,
consider Codahale's
Metric's API (in-band), and JMXtrans (out-of-band).
Noel Milton Vega
Dimension Data, LLC.
[email protected]
[email protected]
On 03/17/2014 08:30 PM, Michael Rose wrote:
Depending on what you need, you could use the Thrift interface to
Nimbus to grab the statistics. The UI project does do some
calculations on the raw metrics, so it's not quite the same. The
algorithms it uses aren't too difficult to replicate.
We ended up building something very similar to what Ooyala did,
customized for our specific needs, it's an excellent pattern.
Michael Rose (@Xorlev <https://twitter.com/xorlev>)
Senior Platform Engineer, FullContact <http://www.fullcontact.com/>
[email protected] <mailto:[email protected]>
On Mon, Mar 17, 2014 at 6:21 PM, Chris Bedford <[email protected]
<mailto:[email protected]>> wrote:
Hi, I'm interested in getting metrics via JMX on not only "
container level " factors (such as # of garbage collects, heap
usage,etc.), but also metrics that describe how spouts and bolts
are performing (e.g., # of tuples emitted, # transferred -- the
same kind of stuff that the storm UI
shows.)
I ran across this project :
:
https://github.com/ooyala/metrics_storm/blob/master/src/ooyala/common/metrics_storm/MetricsStorm.scala
I was just wondering if this is the best best to pursue. Does
anyone have any concrete experience with this that they can share?
thanx!
chris
