Thanks Bobby. If we write our own metrics consumer, how do we ensure that it is better than HttpForwardingMetricsServer? In other words, what aspects of the HttpForwardingMetricsServer should we avoid to make our own metrics consumer better and ready for production?
Is versign/storm-graphite <https://github.com/verisign/storm-graphite> production ready? Also, we should add a line about production-readiness of HttpForwardingMetricsServer in the documentation at http://storm.apache.org/releases/1.0.2/Metrics.html (We were just about to think seriously on using this for production as we thought this to be the standard solution for metrics' consumption in 1.0+ version). -SG On Mon, Oct 10, 2016 at 6:37 AM, Bobby Evans <ev...@yahoo-inc.com> wrote: > First of all there really are two different sets of metrics. One set is > the topology metrics and the other set is the daemon metrics (metrics for > things like the ui and nimbus). The JmxPreparableReporter plugin only > exposes daemon metrics not the topology metrics through JMX. Exposing > topology metrics through JMX is a non trivial task. The current metrics > feature was not designed for this. We are in the process of trying to > redesign the metrics system to allow for features like this, but it is > still a ways off. > > - Bobby > > > On Saturday, October 8, 2016 11:39 AM, S G <sg.online.em...@gmail.com> > wrote: > > > Thanks Bobby, > > We will need some kind of IMetricsConsumer to talk to telegraf. > Many other softwares like Solr, Elastic-Search, Cassandra etc. provide > metrics through a URL making it very easy to consume by tools like telegraf. > How about a IMetricsConsumer that will run on storm-ui and provide the > metrics through a URL such as <storm-ui-host>/metrics ? > > Also, I see the following option in defaults.yaml: > #default storm daemon metrics reporter plugins > storm.daemon.metrics.reporter.plugins: > - "org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter" > > Is this a good option to use for converting metrics into JMX ? > > Thanks > SG > > > > > On Fri, Oct 7, 2016 at 8:11 AM, Bobby Evans <ev...@yahoo-inc.com.invalid> > wrote: > > HttpForwardingMetricsServer is a real hack intended really for tests. I > know I wrote it :). Please don't use it in production. You can write your > own IMetricesConsumer to do whatever you want to with the metrics. > https://github.com/apache/ storm/blob/master/storm-core/ > src/jvm/org/apache/storm/ metric/api/IMetricsConsumer. java > <https://github.com/apache/storm/blob/master/storm-core/src/jvm/org/apache/storm/metric/api/IMetricsConsumer.java> > > That is the correct way to get the data out. If you want to write a > bridge to JMX for this that might work, but going directly to telegraph > would probably be better. - Bobby > > On Thursday, October 6, 2016 1:43 PM, S G <sg.online.em...@gmail.com> > wrote: > > > Hi, > > We want to use Telegraf ( > https://github.com/influxdata/ telegraf/tree/master/plugins > <https://github.com/influxdata/telegraf/tree/master/plugins>) for getting > storm's metrics. > > But we do not want to add a HttpForwardingMetricsServer just to get the > metrics and send them to telegraf. > > Other option is to use Jolokia (https://jolokia.org/) that can read JMX > and > write into telegraf. > > Does storm report all its metrics (including those of custom spouts/bolts) > into JMX? > Or spawning a HttpForwardingMetricsServer is the only option? > > Thanks > SG > > > > > > > >
