Here are the files used to test with lognormalizer. Btw, I'm using rsyslog 8.34.0 from the following repo "deb http://ppa.launchpad.net/adiscon/v8-stable/ubuntu xenial main".
On Tue, May 8, 2018 at 11:19 AM, Rainer Gerhards <[email protected]> wrote: > Not sure if I have time, but maybe others have: can you post both the > file and the rulebase **AS FILES**? I'll at least try to have a look. > > Rainer > > 2018-05-08 10:43 GMT+02:00 Flo Rance <[email protected]>: > > I've tried that as well. Like that: > > > > $ cat postgresql.log > > 2018-05-08 09:30:05.947 CEST [1758] postgres@postgres [local] FATAL: > no > > pg_hba.conf entry for host "[local]", user "postgres", database > "postgres", > > SSL off > > > > And then: > > > > $ /usr/lib/lognorm/lognormalizer -p -r /home/syslog/rules/test.rb < > > postgresql.log > > { "msg": "2018-05-08 09:30:05.947 CEST [1758] postgres@postgres [local] > > FATAL: no pg_hba.conf entry for host \"[local]\", user \"postgres\", > > database \"postgres\", SSL off" } > > > > Honestly, I can't figure out what's wrong. > > > > On Tue, May 8, 2018 at 9:55 AM, Rainer Gerhards < > [email protected]> > > wrote: > >> > >> I am not sure if echo gives you exactly what you think it does. Most > >> importantly, I am not sure if it skips the (important) first space. > >> > >> I usually place the data into a file, so I know exactly what it is. > >> Then I use stdin redirection (<) to put that file into the program, > >> e.g. > >> > >> $ loganalyzer < inputfile .... > >> > >> HTH > >> Rainer > >> > >> 2018-05-08 9:42 GMT+02:00 Flo Rance <[email protected]>: > >> > May anyone give me any tips ? Because I'm completely stucked there. > >> > > >> > I've tried again with a very simple rule: > >> > > >> > version=2 > >> > rule=: %msg:rest% > >> > > >> > It's working correctly with lognormalizer: > >> > > >> > echo ' 2018-05-08 09:30:05.947 CEST [1758] postgres@postgres [local] > >> > FATAL: > >> > no pg_hba.conf entry for host "[local]", user "postgres", database > >> > "postgres", SSL off' | /usr/lib/lognorm/lognormalizer -p -r > >> > /home/syslog/rules/test.rb > >> > { "msg": "2018-05-08 09:30:05.947 CEST [1758] postgres@postgres > [local] > >> > FATAL: no pg_hba.conf entry for host \"[local]\", user \"postgres\", > >> > database \"postgres\", SSL off" } > >> > > >> > But I still get nothing in the debug logs: > >> > > >> > Debug line with all properties: > >> > FROMHOST: 'sc006692.aevisintra.ch', fromhost-ip: '127.0.0.1', > HOSTNAME: > >> > 'sc006692.aevisintra.ch', PRI: 155, > >> > syslogtag 'docker_fluance-authenticationdb[1116]:', programname: > >> > 'docker_fluance-authenticationdb', APP-NAME: > >> > 'docker_fluance-authenticationdb', PROCID: '1116', MSGID: '-', > >> > TIMESTAMP: 'May 8 09:30:05', STRUCTURED-DATA: '-', > >> > msg: ' 2018-05-08 09:30:05.947 CEST [1758] postgres@postgres [local] > >> > FATAL: > >> > no pg_hba.conf entry for host "[local]", user "postgres", database > >> > "postgres", SSL off' > >> > escaped msg: ' 2018-05-08 09:30:05.947 CEST [1758] postgres@postgres > >> > [local] > >> > FATAL: no pg_hba.conf entry for host "[local]", user "postgres", > >> > database > >> > "postgres", SSL off' > >> > inputname: imuxsock rawmsg: '<155>May 8 09:30:05 > >> > docker_fluance-authenticationdb[1116]: 2018-05-08 09:30:05.947 CEST > >> > [1758] > >> > postgres@postgres [local] FATAL: no pg_hba.conf entry for host > >> > "[local]", > >> > user "postgres", database "postgres", SSL off' > >> > $!: > >> > $.: > >> > $/: > >> > > >> > On Fri, May 4, 2018 at 10:35 AM, Flo Rance <[email protected]> > wrote: > >> >> > >> >> I don't really understand what you mean by adding the appropriate > >> >> quotes > >> >> to the rule. They are part of the %msg:rest% at the end. > >> >> > >> >> I've tried again with the single quoted message and I get the > following > >> >> result: > >> >> > >> >> echo ' 2018-05-04 10:24:16.573 CEST [53] postgres@postgres [local] > >> >> FATAL: > >> >> no pg_hba.conf entry for host "[local]", user "postgres", database > >> >> "postgres", SSL off' | /usr/lib/lognorm/lognormalizer -r > >> >> /home/syslog/rules/postgresql.rb > >> >> { "msg": " no pg_hba.conf entry for host \"[local]\", user > >> >> \"postgres\", > >> >> database \"postgres\", SSL off", "severity": "FATAL", "host": > "local", > >> >> "db": > >> >> "postgres", "user": "postgres", "pid": "53", "tz": "CEST", "time": > >> >> "10:24:16.573", "date": "2018-05-04" } > >> >> > >> >> But there's still nothing in the debug logs: > >> >> > >> >> Debug line with all properties: > >> >> FROMHOST: 'sc005827.domain', fromhost-ip: '127.0.0.1', HOSTNAME: > >> >> 'sc005827.domain', PRI: 155, > >> >> syslogtag 'docker_fluance-authenticationdb[1092]:', programname: > >> >> 'docker_fluance-authenticationdb', APP-NAME: > >> >> 'docker_fluance-authenticationdb', PROCID: '1092', MSGID: '-', > >> >> TIMESTAMP: 'May 4 10:24:16', STRUCTURED-DATA: '-', > >> >> msg: ' 2018-05-04 10:24:16.573 CEST [53] postgres@postgres [local] > >> >> FATAL: > >> >> no pg_hba.conf entry for host "[local]", user "postgres", database > >> >> "postgres", SSL off' > >> >> escaped msg: ' 2018-05-04 10:24:16.573 CEST [53] postgres@postgres > >> >> [local] > >> >> FATAL: no pg_hba.conf entry for host "[local]", user "postgres", > >> >> database > >> >> "postgres", SSL off' > >> >> inputname: imuxsock rawmsg: '<155>May 4 10:24:16 > >> >> docker_fluance-authenticationdb[1092]: 2018-05-04 10:24:16.573 CEST > >> >> [53] > >> >> postgres@postgres [local] FATAL: no pg_hba.conf entry for host > >> >> "[local]", > >> >> user "postgres", database "postgres", SSL off' > >> >> $!: > >> >> $.: > >> >> $/: > >> >> > >> >> On Thu, May 3, 2018 at 11:38 PM, David Lang <[email protected]> wrote: > >> >>> > >> >>> your ruleset doesn't have quotes in it. > >> >>> > >> >>> the way you are echoing it with the wrong quotes is causing things > to > >> >>> not > >> >>> work as expected. the line that you are actually processing is: > >> >>> > >> >>> 2018-05-03 11:04:20.201 CEST [24873] postgres@postgres [local] > FATAL: > >> >>> no pg_hba.conf entry for host [local], user postgres, database > >> >>> postgres, SSL > >> >>> off > >> >>> > >> >>> while what is being sent is actually: > >> >>> > >> >>> 2018-05-03 11:04:20.201 CEST [24873] postgres@postgres [local] > FATAL: > >> >>> no pg_hba.conf entry for host "[local]", user "postgres", database > >> >>> "postgres", SSL off > >> >>> > >> >>> add the appropriate quotes to your rule and it should work properly > >> >>> > >> >>> David Lang > >> >>> > >> >>> On Thu, 3 May 2018, Flo Rance wrote: > >> >>> > >> >>>> Date: Thu, 3 May 2018 11:13:02 +0200 > >> >>>> From: Flo Rance <[email protected]> > >> >>>> To: David Lang <[email protected]> > >> >>>> Cc: Rainer Gerhards <[email protected]>, > >> >>>> rsyslog-users <[email protected]> > >> >>>> Subject: Re: [rsyslog] Filter on priority from docker > >> >>>> > >> >>>> Here's the rule that is applied: > >> >>>> > >> >>>> rule=: %date:date-iso% %time:word% %tz:word% [%pid:char-to:\x5d%] > >> >>>> %user:char-to:\x40%@%db:word% [%host:char-to:\x5d%] > >> >>>> %severity:char-to:\x3a%: %msg:rest% > >> >>>> > >> >>>> Here's the output of the lognormalizer utility: > >> >>>> > >> >>>> echo " 2018-05-03 11:04:20.201 CEST [24873] postgres@postgres > [local] > >> >>>> FATAL: no pg_hba.conf entry for host "[local]", user "postgres", > >> >>>> database > >> >>>> "postgres", SSL off" | /usr/lib/lognorm/lognormalizer -r > >> >>>> /home/syslog/rules/postgresql.rb > >> >>>> { "msg": " no pg_hba.conf entry for host [local], user postgres, > >> >>>> database > >> >>>> postgres, SSL off", "severity": "FATAL", "host": "local", "db": > >> >>>> "postgres", > >> >>>> "user": "postgres", "pid": "24873", "tz": "CEST", "time": > >> >>>> "11:04:20.201", > >> >>>> "date": "2018-05-03" } > >> >>>> > >> >>>> and finally the output of rsyslog debug: > >> >>>> > >> >>>> Debug line with all properties: > >> >>>> FROMHOST: 'sc006692.domain', fromhost-ip: '127.0.0.1', HOSTNAME: > >> >>>> 'sc006692.domain', PRI: 155, > >> >>>> syslogtag 'docker_fluance-ehealthdb[1116]:', programname: > >> >>>> 'docker_fluance-ehealthdb', APP-NAME: 'docker_fluance-ehealthdb', > >> >>>> PROCID: > >> >>>> '1116', MSGID: '-', > >> >>>> TIMESTAMP: 'May 3 11:04:20', STRUCTURED-DATA: '-', > >> >>>> msg: ' 2018-05-03 11:04:20.201 CEST [24873] postgres@postgres > [local] > >> >>>> FATAL: no pg_hba.conf entry for host "[local]", user "postgres", > >> >>>> database > >> >>>> "postgres", SSL off' > >> >>>> escaped msg: ' 2018-05-03 11:04:20.201 CEST [24873] > postgres@postgres > >> >>>> [local] FATAL: no pg_hba.conf entry for host "[local]", user > >> >>>> "postgres", > >> >>>> database "postgres", SSL off' > >> >>>> inputname: imuxsock rawmsg: '<155>May 3 11:04:20 > >> >>>> docker_fluance-ehealthdb[1116]: 2018-05-03 11:04:20.201 CEST > [24873] > >> >>>> postgres@postgres [local] FATAL: no pg_hba.conf entry for host > >> >>>> "[local]", > >> >>>> user "postgres", database "postgres", SSL off' > >> >>>> $!: > >> >>>> $.: > >> >>>> $/: > >> >>>> > >> >>>> On Wed, May 2, 2018 at 11:20 PM, David Lang <[email protected]> wrote: > >> >>>> > >> >>>>> Please post your rulebase and the output from RSYSLOG_DebugFormat > so > >> >>>>> that > >> >>>>> we can look at a message that should be matched and what the > ruleset > >> >>>>> for > >> >>>>> the match is. > >> >>>>> > >> >>>>> Odds are that there is something different in the message than you > >> >>>>> think > >> >>>>> it is, so your rule doesn't actually match. > >> >>>>> > >> >>>> > >> >> > >> > > > > > >
2018-05-08 09:30:05.947 CEST [1758] postgres@postgres [local] FATAL: no pg_hba.conf entry for host "[local]", user "postgres", database "postgres", SSL off
postgresql.rb
Description: application/ruby
test.rb
Description: application/ruby
_______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
