On 03/01/2014 10:20 AM, Piotr Czabański wrote:
Hi All, Heka can replacing Statsd - fine. Question: can Heka replace
Graphite?
Sorta, if you squint. We provide a circular buffer library that you can
use in your Lua filters. You store time series data in a cbuf data
structure and emit cbuf messages, and the DashboardOutput will pick them
up and show you real-time updated graphs of your data. If your statsd
data is flowing through Heka, you can convert it to the cbuf format and
build your own graphs from it. You can also add monitoring behavior.
Here's a script that pulls in statsd data about HTTP requests, graphs
them, and does some sliding window standard deviation anomaly detection
on the data, adding annotations to the graph when an anomaly occurs:
http://is.gd/5B4Ygb
And here's the config for the plugin running that script:
http://is.gd/PHdAFu
This only works on the data as it's flowing through Heka, though,
there's no support yet for reading whisperdb files or any other time
series database. And our dashboard UI is still pretty rudimentary. We
plan on putting some cycles into improving that over the next couple of
months.
Is it possible to get udp messages in Statsd standard and go
with it to ElasticSearch with Kibana as a interface?
Sure. If you look at the 'process_message' function in the script I
linked above, you can see code that's extracting statsd data from
message fields and feeding it into cbuf data structures. You could just
as easily be converting the statsd data into json data structures, and
emitting that. Then configure an ElasticSearchOutput to pick the output
messages up and you're done.
I've never tried to use Kibana w/ statsd type time series data before,
so I can't speak to how well it works as compared to graphite, but
getting the data into ES is easy.
I can register some
messages in ElasticSearch placed by Heka - but in "Payload" field there
is few rows and I can not consume it in Kibana.
Hopefully the links above point you in the right direction as to how to
convert statsd data (emitted by Heka's StatAccumulator) to a format that
Kibana can make some sense of. Actually doing so is left as an exercise
to the reader. ;)
-r
_______________________________________________
Heka mailing list
[email protected]
https://mail.mozilla.org/listinfo/heka
