Ok, thank you guys, it's much more obvious. As a side note, CEST is not the programname, but the timezone (Central European Summer Time). So it seems that the programname is missing and only the pid is displayed.
Anyway, I would really like to know how to use this mmnormalize module to parse the message and thus being able to forwared only relevant messages. On Thu, Apr 26, 2018 at 8:22 PM, David Lang <[email protected]> wrote: > On Thu, 26 Apr 2018, Rainer Gerhards wrote: > > 2018-04-26 10:28 GMT+02:00 Flo Rance <[email protected]>: >> >>> Good idea. >>> >>> Here's a typical message that is sent by docker to syslog: >>> >>> Debug line with all properties: >>> FROMHOST: 'sc006692.domain', fromhost-ip: '127.0.0.1', HOSTNAME: >>> 'sc006692.domain', PRI: 155, >>> >> >> OK, rsyslog is doing the right thing. The PRI inside the message (see >> rawmsg below) is 155. Accoding to RFC5424, Sect. 6.2.1 that is a >> facility of 19 and a severity of 3, or according to tables one and two >> in that section local3 with error severity. >> > > so the docker log-driver is setting the severity to error > > syslogtag 'docker_fluance-ehealthdb[1116]:', programname: >>> 'docker_fluance-ehealthdb', APP-NAME: 'docker_fluance-ehealthdb', PROCID: >>> '1116', MSGID: '-', >>> TIMESTAMP: 'Apr 26 10:22:45', STRUCTURED-DATA: '-', >>> msg: ' 2018-04-26 10:22:45.283 CEST [13518] postgres@postgres [local] >>> FATAL: >>> no pg_hba.conf entry for host "[local]", user "postgres", database >>> "postgres", SSL off' >>> >> > But in the message itself, it is providing different information, > including a timestamp (2018-04-26 10:22:45.283) programname (CEST) pid > (13518) and finally severity (FATAL:) > > escaped msg: ' 2018-04-26 10:22:45.283 CEST [13518] postgres@postgres >>> [local] FATAL: no pg_hba.conf entry for host "[local]", user "postgres", >>> database "postgres", SSL off' >>> inputname: imuxsock >>> >> > rawmsg: '<155>Apr 26 10:22:45 >>> docker_fluance-ehealthdb[1116]: 2018-04-26 10:22:45.283 CEST [13518] >>> postgres@postgres [local] FATAL: no pg_hba.conf entry for host >>> "[local]", >>> user "postgres", database "postgres", SSL off' >>> >> > If you look at the rawmsg, which is what the docker log-driver passes to > rsyslog, you will see that it sets the pri to 155, the timestamp to Apr 26 > 10:22:45 and the programname/pid to docker_fluance-ehealthdb[1116] > > the docker log driver makes the _assumption_ that what is spit out to > stdout is not structured, so it adds a default header and passes it to > rsyslog. > > I have to run to a meeting, I'll post some info later for configuring > mmnormalize to parse the message. > > David Lang > > > $!: >>> $.: >>> $/: >>> >>> On Thu, Apr 26, 2018 at 9:57 AM, Rainer Gerhards < >>> [email protected]> >>> wrote: >>> >>>> >>>> 2018-04-26 9:48 GMT+02:00 Flo Rance via rsyslog >>>> <[email protected]>: >>>> >>>>> What do you mean by "not in any standard format" ? >>>>> I've explicitely declared the logging driver in docker to be syslog, so >>>>> I >>>>> was expecting to be able to parse the result to extract accurate data >>>>> in >>>>> the fields. >>>>> >>>>> Can you give me any hints on how I could use mmnormalize to extract the >>>>> various fields ? >>>>> >>>> >>>> let's get started with something easier. Please add >>>> >>>> *.* /var/log/messagedebug;RSYSLOG_DebugFormat >>>> >>>> to the top of your rsyslog.conf. Let some messages flow. In the new >>>> file, you should then see that message together with the decoded >>>> properties AND the raw message. Post that entry (~6 lines or so). With >>>> that, we know for sure what is going on. >>>> >>>> Rainer >>>> >>>> >>>>> >>>>> On Wed, Apr 25, 2018 at 7:28 PM, David Lang <[email protected]> wrote: >>>>> >>>>> On Wed, 25 Apr 2018, Rainer Gerhards wrote: >>>>>> >>>>>> 2018-04-25 9:29 GMT+02:00 Flo Rance <[email protected]>: >>>>>> >>>>>>> >>>>>>> Ok, but if ".err" means "err and above", why does it forward messages >>>>>>>> with >>>>>>>> the severity INFO as in the example ? >>>>>>>> >>>>>>>> >>>>>>> pls post the raw message - how do you know it is INFO? >>>>>>> >>>>>>> >>>>>> in the docker world, the 'standard' is that messages get dumped to >>>>>> stdout, >>>>>> not in any standard format, so INFO: in the message body is the >>>>>> indication. >>>>>> >>>>>> It looks like these logs should be parsed with mmnormalize to extract >>>>>> the >>>>>> various fields (potentially as a parser on the input) >>>>>> >>>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com/professional-services/ >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>> myriad >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>> DON'T >>>>> LIKE THAT. >>>>> >>>> >>> >>> >> _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
