In order to get this / have the metrics consumer work, do I need to have
the setDebug attribute set to true?
On Mon, Sep 22, 2014 at 12:59 PM, Harsha <st...@harsha.io> wrote:
> Here is what I see in the metrics.log
>
> 2014-09-22 09:44:31,321 73175 1411404271 localhost:6703
> 19:split __transfer-count {default=2680}
>
> 2014-09-22 09:44:31,321 73175 1411404271 localhost:6703
> 19:split __execute-latency {spout:default=0.0}
>
> 2014-09-22 09:44:31,321 73175 1411404271 localhost:6703
> 19:split __fail-count {}
>
> 2014-09-22 09:44:31,321 73175 1411404271 localhost:6703
> 19:split __emit-count {default=2680}
>
> 2014-09-22 09:44:31,321 73175 1411404271 localhost:6703
> 19:split __execute-count {spout:default=420}
>
> 2014-09-22 09:44:31,352 73179 1411404271 localhost:6703
> 22:split __ack-count {spout:default=420}
>
> 2014-09-22 09:44:31,352 73179 1411404271 localhost:6703
> 22:split __sendqueue {write_pos=2679, capacity=1024,
> read_pos=2679, population=0}
> I do see all the UI related counts coming in the metrics.log.
>
> -Harsha
>
>
> On Mon, Sep 22, 2014, at 10:41 AM, Raphael Hsieh wrote:
>
> Hi Harsha,
> Did you have to bind the metrics consumer to the default StormUI metrics
> at all? Or do those automagically get included ?
>
> Thanks!
>
> On Mon, Sep 22, 2014 at 10:33 AM, Otis Gospodnetic <
> otis.gospodne...@gmail.com> wrote:
>
> Hi Gezim,
>
> On Fri, Sep 19, 2014 at 7:27 PM, Gezim Musliaj <gmusl...@gmail.com> wrote:
>
> Hey Otis, I was just registered at sematext and I can say that this is
> what I have been looking for.I have just one question, what about the
> delays between the SPM and the Storm Cluster (if they do exist), whats the
> worst case? I mean because these metrics are not calculated locally, but
> using an internet connection.
>
>
>
> The worst case is that somebody unplugs your servers from the network, but
> if that happens you have bigger problems to deal with. In all seriousness,
> Storm (local) => SPM (remote/cloud/saas) is not really a problem -- lots of
> people successfully use SPM for monitoring Storm, Hadoop, Kafka, and other
> types of systems.
>
> Otis
> --
> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
> Solr & Elasticsearch Support * http://sematext.com/
>
>
>
>
> Thanks !
>
> On Sat, Sep 20, 2014 at 1:15 AM, Otis Gospodnetic <
> otis.gospodne...@gmail.com> wrote:
>
> Raphael,
>
> Not sure if this is what you are after, but SPM <http://sematext.com/spm/>
> will collect and graph all Storm metrics, let you do alerting and anomaly
> detection on them, etc. If you want to graph custom metrics (e.g.
> something from your bolts), you can send them in as custom metrics
> <https://sematext.atlassian.net/wiki/display/PUBSPM/Custom+Metrics> and
> again graph them, alert on them, do anomaly detection on them, stick them
> on dashboards, etc. If you want to emit events from your bolts, you can send
> events to SPM
> <https://sematext.atlassian.net/wiki/display/PUBSPM/Events+Integration>,
> too, or you can send them to Logsene <http://www.sematext.com/logsene/>...
> can be handy for correlation with alerts and performance graphs when
> troubleshooting. Here are some Storm metrics graph:
> http://blog.sematext.com/2014/01/30/announcement-apache-storm-monitoring-in-spm/
>
> I hope this helps.
>
> Otis
> --
> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
> Solr & Elasticsearch Support * http://sematext.com/
>
>
> On Fri, Sep 19, 2014 at 6:12 PM, Raphael Hsieh <raffihs...@gmail.com>
> wrote:
>
> Hi,
> Using Storm/Trident, how do I register a metrics consumer to log the data
> I get in the StormUI ?
> I want to look at historical data of my topology, for example the execute
> latency of the topology over time, as this would give me good insight as to
> where things might be going wrong when the system breaks.
>
> I have been following the steps outlined in the BigData CookBook here:
> http://www.bigdata-cookbook.com/post/72320512609/storm-metrics-how-to
>
> However I am not wanting to create my own metrics, instead I just want to
> log the metrics that already exist built in to Storm. It is unclear to me
> how I am supposed to go about doing that.
>
> Thanks
>
> --
> Raphael Hsieh
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
> Raphael Hsieh
>
>
>
>
>
>
--
Raphael Hsieh
