It is not necessary to have debug enabled to get metrics. Also, it is
independent of storm ui. The ui does not need to be running - the metrics
are reported by the workers.
On Wed, Sep 24, 2014 at 12:12 PM, Raphael Hsieh <raffihs...@gmail.com>
wrote:
> In order to get this / have the metrics consumer work, do I need to have
> the setDebug attribute set to true?
>
> On Mon, Sep 22, 2014 at 12:59 PM, Harsha <st...@harsha.io> wrote:
>
>> Here is what I see in the metrics.log
>>
>> 2014-09-22 09:44:31,321 73175 1411404271 localhost:6703
>> 19:split __transfer-count {default=2680}
>>
>> 2014-09-22 09:44:31,321 73175 1411404271 localhost:6703
>> 19:split __execute-latency {spout:default=0.0}
>>
>> 2014-09-22 09:44:31,321 73175 1411404271 localhost:6703
>> 19:split __fail-count {}
>>
>> 2014-09-22 09:44:31,321 73175 1411404271 localhost:6703
>> 19:split __emit-count {default=2680}
>>
>> 2014-09-22 09:44:31,321 73175 1411404271 localhost:6703
>> 19:split __execute-count {spout:default=420}
>>
>> 2014-09-22 09:44:31,352 73179 1411404271 localhost:6703
>> 22:split __ack-count {spout:default=420}
>>
>> 2014-09-22 09:44:31,352 73179 1411404271 localhost:6703
>> 22:split __sendqueue {write_pos=2679, capacity=1024,
>> read_pos=2679, population=0}
>> I do see all the UI related counts coming in the metrics.log.
>>
>> -Harsha
>>
>>
>> On Mon, Sep 22, 2014, at 10:41 AM, Raphael Hsieh wrote:
>>
>> Hi Harsha,
>> Did you have to bind the metrics consumer to the default StormUI metrics
>> at all? Or do those automagically get included ?
>>
>> Thanks!
>>
>> On Mon, Sep 22, 2014 at 10:33 AM, Otis Gospodnetic <
>> otis.gospodne...@gmail.com> wrote:
>>
>> Hi Gezim,
>>
>> On Fri, Sep 19, 2014 at 7:27 PM, Gezim Musliaj <gmusl...@gmail.com>
>> wrote:
>>
>> Hey Otis, I was just registered at sematext and I can say that this is
>> what I have been looking for.I have just one question, what about the
>> delays between the SPM and the Storm Cluster (if they do exist), whats the
>> worst case? I mean because these metrics are not calculated locally, but
>> using an internet connection.
>>
>>
>>
>> The worst case is that somebody unplugs your servers from the network,
>> but if that happens you have bigger problems to deal with. In all
>> seriousness, Storm (local) => SPM (remote/cloud/saas) is not really a
>> problem -- lots of people successfully use SPM for monitoring Storm,
>> Hadoop, Kafka, and other types of systems.
>>
>> Otis
>> --
>> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
>> Solr & Elasticsearch Support * http://sematext.com/
>>
>>
>>
>>
>> Thanks !
>>
>> On Sat, Sep 20, 2014 at 1:15 AM, Otis Gospodnetic <
>> otis.gospodne...@gmail.com> wrote:
>>
>> Raphael,
>>
>> Not sure if this is what you are after, but SPM
>> <http://sematext.com/spm/> will collect and graph all Storm metrics, let
>> you do alerting and anomaly detection on them, etc. If you want to graph
>> custom metrics (e.g. something from your bolts), you can send them in as
>> custom
>> metrics
>> <https://sematext.atlassian.net/wiki/display/PUBSPM/Custom+Metrics> and
>> again graph them, alert on them, do anomaly detection on them, stick them
>> on dashboards, etc. If you want to emit events from your bolts, you can send
>> events to SPM
>> <https://sematext.atlassian.net/wiki/display/PUBSPM/Events+Integration>,
>> too, or you can send them to Logsene <http://www.sematext.com/logsene/>...
>> can be handy for correlation with alerts and performance graphs when
>> troubleshooting. Here are some Storm metrics graph:
>> http://blog.sematext.com/2014/01/30/announcement-apache-storm-monitoring-in-spm/
>>
>> I hope this helps.
>>
>> Otis
>> --
>> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
>> Solr & Elasticsearch Support * http://sematext.com/
>>
>>
>> On Fri, Sep 19, 2014 at 6:12 PM, Raphael Hsieh <raffihs...@gmail.com>
>> wrote:
>>
>> Hi,
>> Using Storm/Trident, how do I register a metrics consumer to log the data
>> I get in the StormUI ?
>> I want to look at historical data of my topology, for example the execute
>> latency of the topology over time, as this would give me good insight as to
>> where things might be going wrong when the system breaks.
>>
>> I have been following the steps outlined in the BigData CookBook here:
>> http://www.bigdata-cookbook.com/post/72320512609/storm-metrics-how-to
>>
>> However I am not wanting to create my own metrics, instead I just want to
>> log the metrics that already exist built in to Storm. It is unclear to me
>> how I am supposed to go about doing that.
>>
>> Thanks
>>
>> --
>> Raphael Hsieh
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> --
>> Raphael Hsieh
>>
>>
>>
>>
>>
>>
>
>
>
> --
> Raphael Hsieh
>
>
>
>
