Here is what I see in the metrics.log
2014-09-22 09:44:31,321 73175 1411404271
localhost:6703 19:split __transfer-count
{default=2680}
2014-09-22 09:44:31,321 73175 1411404271
localhost:6703 19:split __execute-latency
{spout:default=0.0}
2014-09-22 09:44:31,321 73175 1411404271
localhost:6703 19:split __fail-count {}
2014-09-22 09:44:31,321 73175 1411404271
localhost:6703 19:split __emit-count
{default=2680}
2014-09-22 09:44:31,321 73175 1411404271
localhost:6703 19:split __execute-count
{spout:default=420}
2014-09-22 09:44:31,352 73179 1411404271
localhost:6703 22:split __ack-count
{spout:default=420}
2014-09-22 09:44:31,352 73179 1411404271
localhost:6703 22:split __sendqueue
{write_pos=2679, capacity=1024, read_pos=2679, population=0}
I do see all the UI related counts coming in the metrics.log.
-Harsha
On Mon, Sep 22, 2014, at 10:41 AM, Raphael Hsieh wrote:
Hi Harsha,
Did you have to bind the metrics consumer to the default
StormUI metrics at all? Or do those automagically get included
?
Thanks!
On Mon, Sep 22, 2014 at 10:33 AM, Otis Gospodnetic
<[1]otis.gospodne...@gmail.com> wrote:
Hi Gezim,
On Fri, Sep 19, 2014 at 7:27 PM, Gezim Musliaj
<[2]gmusl...@gmail.com> wrote:
Hey Otis, I was just registered at sematext and I can say that
this is what I have been looking for.I have just one question,
what about the delays between the SPM and the Storm Cluster (if
they do exist), whats the worst case? I mean because these
metrics are not calculated locally, but using an internet
connection.
The worst case is that somebody unplugs your servers from the
network, but if that happens you have bigger problems to deal
with. In all seriousness, Storm (local) => SPM
(remote/cloud/saas) is not really a problem -- lots of people
successfully use SPM for monitoring Storm, Hadoop, Kafka, and
other types of systems.
Otis
--
Monitoring * Alerting * Anomaly Detection * Centralized Log
Management
Solr & Elasticsearch Support * [3]http://sematext.com/
Thanks !
On Sat, Sep 20, 2014 at 1:15 AM, Otis Gospodnetic
<[4]otis.gospodne...@gmail.com> wrote:
Raphael,
Not sure if this is what you are after, but [5]SPM will collect
and graph all Storm metrics, let you do alerting and anomaly
detection on them, etc. If you want to graph custom metrics
(e.g. something from your bolts), you can send them in as
[6]custom metrics and again graph them, alert on them, do
anomaly detection on them, stick them on dashboards, etc. If
you want to emit events from your bolts, you can [7]send events
to SPM, too, or you can send them to [8]Logsene... can be handy
for correlation with alerts and performance graphs when
troubleshooting. Here are some Storm metrics graph:
[9]http://blog.sematext.com/2014/01/30/announcement-apache-stor
m-monitoring-in-spm/
I hope this helps.
Otis
--
Monitoring * Alerting * Anomaly Detection * Centralized Log
Management
Solr & Elasticsearch Support * [10]http://sematext.com/
On Fri, Sep 19, 2014 at 6:12 PM, Raphael Hsieh
<[11]raffihs...@gmail.com> wrote:
Hi,
Using Storm/Trident, how do I register a metrics consumer to
log the data I get in the StormUI ?
I want to look at historical data of my topology, for example
the execute latency of the topology over time, as this would
give me good insight as to where things might be going wrong
when the system breaks.
I have been following the steps outlined in the BigData
CookBook
here: [12]http://www.bigdata-cookbook.com/post/72320512609/stor
m-metrics-how-to
However I am not wanting to create my own metrics, instead I
just want to log the metrics that already exist built in to
Storm. It is unclear to me how I am supposed to go about doing
that.
Thanks
--
Raphael Hsieh
--
Raphael Hsieh
References
1. mailto:otis.gospodne...@gmail.com
2. mailto:gmusl...@gmail.com
3. http://sematext.com/
4. mailto:otis.gospodne...@gmail.com
5. http://sematext.com/spm/
6. https://sematext.atlassian.net/wiki/display/PUBSPM/Custom+Metrics
7. https://sematext.atlassian.net/wiki/display/PUBSPM/Events+Integration
8. http://www.sematext.com/logsene/
9.
http://blog.sematext.com/2014/01/30/announcement-apache-storm-monitoring-in-spm/
10. http://sematext.com/
11. mailto:raffihs...@gmail.com
12. http://www.bigdata-cookbook.com/post/72320512609/storm-metrics-how-to
