On 04/06/2015 02:59 PM, Tom Davis wrote:
I'm using Heka as a generic pipeline for source code packaging. For instance, one input is a URL that gets "decoded" to source code (tarballs are downloaded and extracted, git repos cloned, etc.). The source is "filtered" in Lua based on matches (name, source language, whatever); this may involve invoking external processes, adding new metadata files to the source tree, etc. When everything is in order, the new tree is "encoded" in some way (packaged as RPM, Docker container, etc.) and output.
Huh, wow. Yeah, that *is* pretty different than most of the things I've seen Heka used for. Neat!
For obvious reasons I'm not pushing the raw bytes of arbitrary source trees through messages, so a lot of the transforming happens on the file system. The messages carry the metadata filters need to match and/or decide what to do. If a particular project needs custom logic (and they often do) I can load custom-built filters easily through the sandbox manager. Messages are also an easy way to indirectly communicate between parts of the pipeline. An abuse, perhaps, but no sense bolting on more channels.
For the record, I don't consider this an abuse. Messages are explicitly meant to be used for, well, for whatever you decide to use them for.
My steps don't always map super cleanly to parts of the Heka pipeline, but this beats writing and instrumenting my own script-able state machine. Thanks for the interest!
Glad you were able to bend it to your will, and glad you're finding it useful. :)
-r _______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
