[rsyslog] Imkafka problem on topic subscription

Dear community, sorry if I'm re-posting here the GitHub issue https://github.com/rsyslog/rsyslog/issues/1771 , I'm try to reach other people that maybe had my same problem. I'trying to build a centralized logging system using: omkafka -> kafka -> imkafka -> omfile The rsyslog sender

[rsyslog] omfile - create file path hierarchy splitting hostname

Hello group, my current server hostname looks like ---. Where environment can be: stg,stg2.. etc.; serverType: api,worker.. etc and server ID is a unique ID. I'm trying to configure a receiver rsyslog server that is automatically able to split the hostname and write the log file in a hierarchy

Re: [rsyslog] omfile - create file path hierarchy splitting hostname

@David thank you I made it works without mmnormalize, just using rsyslog functions, here is the snippet for somebody reference: module(load="imkafka") input(type="imkafka" topic="logging" broker=["IP"] ruleset="pRuleset" consumergroup="default1" confParam=["compression.codec=snappy",

Re: [rsyslog] omfile - create file path hierarchy splitting hostname

I agree with you David, I tried mmnormalize but I wasn't able to make it work. I was assigning to all the local variables (for example: set $.environment = $!environment ) the variables extracted by mmnormalize and they were always empty. I checked my rule using lognormalizer and the output field

Re: [rsyslog] omfile - create file path hierarchy splitting hostname

Hello David, this is what I was trying to accomplish: *example of the logline:* Oct 2 22:44:00 staging3-worker-i-0a646drtgbfb4dbe35 serverID:i-0a646drtgbfb4dbe35 2017-10-02 22:43:55.144 INFO 22940 --- [yBean_Worker-17] com.domain.jobs.class.Job : Job completed, jobId: 26, alerts: 2,

Re: [rsyslog] omfile - create file path hierarchy splitting hostname

The file is in / no selinux or apparmor enabled. On Oct 9, 2017 7:28 PM, "David Lang" wrote: > On Mon, 9 Oct 2017, Luigi Tagliamonte wrote: > > ruleset(name="pRuleset") { >>action(type="mmnormalize" rulebase="/rule") >> > > is the file really in /? > > for many distros,

Re: [rsyslog] omfile - create file path hierarchy splitting hostname

hat the actual msg properties look like, and pass that into > the lognormalizer tester. > > That's probably something that should explicitely be mentioned in the doc. > > Rainer > > 2017-10-09 18:10 GMT+02:00 Luigi Tagliamonte via rsyslog > <rsyslog@lists.adiscon.

Re: [rsyslog] Help select a new logo

How to vote? On Thu, Feb 1, 2018 at 10:15 AM, Rainer Gerhards wrote: > ... as said earlier today, we would appreciate if you could let us > know what you like best: > > http://www.rsyslog.com/help-select-a-logo/ > > Thanks to everyone! > Rainer >

Re: [rsyslog] rsyslog 8.33.0 (v8-stable) released

@florian does this release contain also a new build of im/omkafka modules including this fix: link ? On Tue, Feb 20, 2018 at 6:59 AM, Florian Riedl wrote: > Hi all, > > Today, we release rsyslog

Re: [rsyslog] rsyslog 8.33.0 (v8-stable) released

we > provide are built with the v0.11.3 release from Dec 4. > > 2018-02-20 17:10 GMT+01:00 Luigi Tagliamonte via rsyslog > <rsyslog@lists.adiscon.com>: > > @florian does this release contain also a new build of im/omkafka modules > > including this fix: link > >

Re: [rsyslog] rsyslog 8.33.0 (v8-stable) released

you build librdkafka yourself on your > >> system from the most recent sourcecode available. The packages we > >> provide are built with the v0.11.3 release from Dec 4. > >> > >> 2018-02-20 17:10 GMT+01:00 Luigi Tagliamonte via rsyslog > >> <rsyslog@lis

[rsyslog] Imkafka/omkafka tuning

Hi there! What are the tunable parameters for this module, like: - an option to increase the number of threads for kafka processing - number of messages to process per req. - etc.. Regards L. ___ rsyslog mailing list

Re: [rsyslog] what's the best way of getting logs out of docker (kubernetes)

I personally think that the best way is to have the docker log kafka plugin sending logs to Kafka and then fetch then with rsyslog Kafka in module. On Wed, Mar 28, 2018, 5:47 PM David Lang wrote: > sending the logs through journald involves a LOT of wasted effort > > writing the

Re: [rsyslog] what's the best way of getting logs out of docker (kubernetes)

I do use Kafka for centralize all the logging and messaging so was a no brain choice for me. Plus gives you durability in case you have problems with syslog/indexing services. On Wed, Mar 28, 2018, 6:13 PM David Lang <da...@lang.hm> wrote: > On Thu, 29 Mar 2018, Luigi Tagliamonte vi