Rob Miller writes: > On 04/06/2015 02:59 PM, Tom Davis wrote: >> I'm using Heka as a generic pipeline for source code packaging. For >> instance, one input is a >> URL that gets "decoded" to source code (tarballs are downloaded and >> extracted, git repos cloned, >> etc.). The source is "filtered" in Lua based on matches (name, source >> language, whatever); this may >> involve invoking external processes, adding new metadata files to the source >> tree, etc. When >> everything is in order, the new tree is "encoded" in some way (packaged as >> RPM, Docker container, >> etc.) and output. > Huh, wow. Yeah, that *is* pretty different than most of the things I've seen > Heka used for. Neat! >> For obvious reasons I'm not pushing the raw bytes of arbitrary source trees >> through messages, so a >> lot of the transforming happens on the file system. The messages carry the >> metadata filters need to >> match and/or decide what to do. If a particular project needs custom logic >> (and they often do) I can >> load custom-built filters easily through the sandbox manager. Messages are >> also an easy way to >> indirectly communicate between parts of the pipeline. An abuse, perhaps, but >> no sense bolting on >> more channels. > For the record, I don't consider this an abuse. Messages are explicitly meant > to be used for, well, for whatever you decide to use them for. >> My steps don't always map super cleanly to parts of the Heka pipeline, but >> this beats writing and >> instrumenting my own script-able state machine. Thanks for the interest! >> > Glad you were able to bend it to your will, and glad you're finding it > useful. :)
Me too! It took me a while to create a reasonable build story (a little ironic given the application I guess); deploying Heka with an existing package is one thing, integrating with its build/package processes to write my own Heka-based application without forking the project was another. Anyway, I'm confident it was a good investment given how responsive and active you and trink have been with the project and via your support here on the list. Keep up the great work! :) > > > -r _______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
