Thanks a lot for your answers. And yes, I'm very interested in bootstrapping a first prototype of my own data pipeline based on Hindsight so that I can compare the performance on a raspberry pi. (here is the current state of our Heka-based data pipeline: https://bitbucket.org/helioslite/heka-hl-sandboxes) So it would be great if you can give me the first instructions on how to build and setup Hindsight.
Thanks. Bruno On 10 June 2015 at 18:18, Michael Trinkala <[email protected]> wrote: > - It is usable and being actively developed with the intent to move it > into production later this year. > - We are currently running production data through it for testing but it > is not deployed in an official capacity. It has been very stable but until > a more robust set of tests have been build out I will not consider it > production ready. > - Yes, it can decode/encode Heka protobuf format > - Yes, the router/message matcher is complete. The only difference is > that it supports Lua string pattern matching instead of re2 regexp (Heka > 'Hostname =~ /^foo/' vs Hindsight 'Hostname =~ "^foo"') > - Yes, but you would need a lua-socket input and output sandbox (see > benchmarks/hsr_run for related examples) > - No documentation yet, only examples in the benchmarks directory. I > could have you bootstrapped in about 30 minutes (and hopefully turn that > into a getting started guide) if you are interested. > > Implementation wise the only missing piece is support for dynamically > loading plugins. The actual code to accomplish it is very small (just > detecting files in the load directory and moving them to the run directory) > but ideally it would be fronted by a web server and a GUI with access > control and validation (a much larger effort and actually a separate > project). > > Trink > > On Wed, Jun 10, 2015 at 8:15 AM, bruno binet <[email protected]> > wrote: > >> Hi, >> >> I recently discovered the work pushed into the Hindsight repository ( >> https://github.com/trink/hindsight) which seems to be a lightweight >> alternative to Heka, based on the lua sandbox. >> The Hindsight vs Heka benchmarks are quite impressive. >> >> I'm currently running Heka on the raspberry pi (not so powerful) device >> and the load average quickly increases and exceeds 1 when Heka is ingesting >> data, so Hindsight could be a good fit for us if it can perform better than >> Heka in terms of CPU cycles. >> >> What is the current status of Hindsight? Is it just an temporary >> experiment or will it be maintained and actually used in production? >> Is it currently usable and stable? >> Is Hindsight able to decode and encode Heka protobuf format? >> Does Hindsight have a complete router implementation to dispatch messages >> to sandboxes like in Heka? >> My use case is basically to read raw text data from UDP socket, parse >> text data with lua patterns or lpeg, process data through a few lua sandbox >> filters, then write output messages both to a file (protobuf heka format) >> and a HTTP server (json format): can this be easily accomplished with >> Hindsight? >> Is there any documentation somewhere to get started with Hindsight? >> >> Thanks, >> Bruno >> >> _______________________________________________ >> Heka mailing list >> [email protected] >> https://mail.mozilla.org/listinfo/heka >> >> >
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
