This sounds like something that could easily be solved with a configuration management solution that builds Heka's configuration from templates based on logic defined within it. You could check whether the host being deployed to has the directory you are wanting to check for the existence of and then define conditional statements to include or exclude the segments in Heka's configuration that define Logstreamer inputs for them. Since heka can include all files within a given directory, you could also optionally include the entire file that defines the Logstreamer configuration. I have had much success with something similar using Ansible and the Jinja2 templating that it utilizes. This would, of course, avoid changes to the Heka code altogether.
On Wed, Jan 6, 2016 at 7:10 AM Eric LEMOINE <[email protected]> wrote: > On Tue, Jan 5, 2016 at 12:09 AM, Rob Miller <[email protected]> wrote: > > I could maybe be convinced, but I'm leaning negative on this one. > > Logstreamer is already a pretty complicated feature, with a lot of fiddly > > code. I'm loathe to add yet more features / complexity unless the win is > > big. It doesn't seem to terribly unreasonable for users to create an > empty > > folder on the machine in question. > > Thank you Rob for your response. > > Have you looked at my patch [1]? From my perspective the patch brings > no additional complexity to Logstreamer. It just moves two function > calls from the NewLogstreamSet function to the ScanForLogstreams > function. And I think it does bring flexibility to users. In my case, > to reduce configuration management complexity, I'd like to deploy the > same Heka configuration on all the nodes of my data center. For this > to work I'd need Logstreamer to be adaptive, and to not fail when the > log_directory doesn't exist. After all Logstreamer already doesn't > fail when there are no files matching file_match, so I don't quite > understand why it should fail when log_directory does not exist. > Please tell me if I'm not viewing this from the proper angle. > > Thanks! > > [1] < > https://github.com/mozilla-services/heka/compare/dev...elemoine:logstreamer > > > _______________________________________________ > Heka mailing list > [email protected] > https://mail.mozilla.org/listinfo/heka > -- Regards, Tony Cesaro [email protected]
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
