You can also take a look at storm-perf-test ( https://github.com/yahoo/storm-perf-test/) source code. I'm currently trying to extract some metrics, in order to develop benchmarks for storm and other stream processors, and I thought it was really useful.
Thanks, Otávio. Undergraduate Student at Federal University of Rio Grande do Sul - http://inf.ufrgs.br Scholarship holder at Parallel and Distributed Processing Group - http://gppd.inf.ufrgs.br [email protected] / @otaviocarvalho 2014-02-24 12:50 GMT-03:00 Milinda Pathirage <[email protected]>: > Hi Padma, > > I think answers to your questions are there in the article you > mentioned. Anyway I'll try to explain what needs to be done briefly. > Note that I don't have any experience on statsd or graphite. > > First on sumerizing the metrics in metrics.log file. If you want to > summerize the metrics mentioned in the article, you will have to write > your own summarizer. It depends on what and how to summerize data you > collected. It looks like fields in metrics.log contains information > such as timestamp of the metrics publish event, storm host name, bolt > identifier, metrics identifier and actual metrics value. You should be > able to understand it by reading [2]. > > - It looks like people are using statsd to feed graphite[1]. And > author of the article you mentioned also planning to use the same > approach. > - In this case you need to first write a metrics consumer which > publish metrics to statsd. > - Then connnect statsd and graphite according to [2]. > > I think its possible to write a metrics consumer which directly feed > graphite. But I am not sure whether which approach is easier. > > Thanks > Milinda > > [1] > http://matt.aimonetti.net/posts/2013/06/26/practical-guide-to-graphite-monitoring/ > [2] > https://github.com/nathanmarz/storm/blob/master/storm-core/src/jvm/backtype/storm/metric/LoggingMetricsConsumer.java > > On Mon, Feb 24, 2014 at 8:40 AM, padma priya chitturi > <[email protected]> wrote: > > Hi All, > > > > I've been using storm metrics to visualize the performance of storm ( > > http://www.bigdata-cookbook.com/post/72320512609/storm-metrics-how-to). > > > > I have included the metrics initialization code in ExclamationTopology > code > > and saw the metrics in metrics.log > > > > How can we summarize the metrics in metrics.log file. ? what do different > > fields mean ? How can we visualize the metrics using Graphite ? > > > > Can someone suggest me in this.. > > > > Thanks, > > Padma Ch. > > > > -- > Milinda Pathirage > > PhD Student | Research Assistant > School of Informatics and Computing | Data to Insight Center > Indiana University > > twitter: milindalakmal > skype: milinda.pathirage > blog: http://milinda.pathirage.org >
