on the receiver, write a log with the format rawmsg or use the RSYSLOG_DebugFormat and look at the rawmsg line there. Let's see exactly what is being sent to see if the data is being lost at transmit or on reception.

Personally, I have my senders reformat the data so that the body of the message is in JSON, and then I create a $!trusted variable that I have contain various metadata. each machine that processes the message adds to the metadata (when it was received, what machine recived it, what IP it was recevied from,etc)


This is a good place to put high res timing info or syslogtag info to make sure that nothing mangles it in processing. On the final receiver, you then parse the JSON and have access to everything you every want.

This is an indirect way of solving the problem you have, but it solves a bunch of other problems at the same time, and doesn't require the SyslogProtocol123Format to get the timestamp you want (you may even just include the timestamp in various formats so you don't have to mess with changing it's format later)

David Lang

 On Sat, 5 Aug 2017, deoren wrote:

Date: Sat, 5 Aug 2017 22:59:59 -0500
From: deoren <rsyslog-users-lists.adiscon....@whyaskwhy.org>
Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
To: rsyslog@lists.adiscon.com
Subject: [rsyslog] How can I use high precision forwarding format,
    but still preserve the Postfix process name in forwarded messages?

I've recently converted all of our nodes from forwarding messages from the default forwarding format to using the 'RSYSLOG_SyslogProtocol23Format' format.

I only did light research beforehand (so I can only blame myself), but when our relay nodes log in either 'RSYSLOG_TraditionalFileFormat' or 'RSYSLOG_FileFormat' the process name is recorded in the log file(s). When we forward messages using the default forwarding format, that information seems to come across as expected. When our client nodes forward using the 'RSYSLOG_SyslogProtocol23Format' format the process name information is "lost" and not recorded within the local log files on our receiver.

DETAILS

sender:

* rsyslog 8.28.0
* Ubuntu 16.04
* using official PPA
* Postfix mail relay node
* sending in 'RSYSLOG_SyslogProtocol23Format' format via RELP
* recording locally in 'RSYSLOG_FileFormat' format
* local messages appear to retain 'postfix/smtpd[9999]' format (which is desired)

receiver:

* rsyslog 8.28.0
* Ubuntu 16.04
* using official PPA
* receiving via RELP
* recording locally in 'RSYSLOG_FileFormat' format
* messages saved appear to no longer retain the process name (e.g., 'postfix[9999]')


Turning to Google, I found this[1] bug report which explained the problem much better than I ever could. I later found another thread[2] where the problem was discussed, but no resolution reached. The final post to that thread[4] illustrated that the syslogtag value is coming across with the values needed in the syslogtag field, but both programname and app-name properties were missing the Postfix component/process name information.

Is there a way to subsclass or inherit the RSYSLOG_FileFormat template and override the value it uses for app-name? I don't know what I am doing in that regarding, but I'm willing to learn. I'd really like to preserve the Postfix process name in the log messages saved on our receiver node.

In absence of the understanding necessary to override the existing template, one workaround I'm considering is having our client nodes check the facility before forwarding messages on to the central receiver. If the facility is 'mail', send using traditional forwarding format, otherwise send everything else using the newer forwarding format. I'd rather have precision timestamps on both, but having the process name available in the mail logs is more important in our use case.

Thank you in advance for your help!


References:

[1] https://github.com/rsyslog/rsyslog/issues/168

[2] http://kb.monitorware.com/app-name-programname-t12945.html

[3] http://kb.monitorware.com/post27055.html#p27055

[4] http://kb.monitorware.com/post27061.html#p27061

[5] http://www.rsyslog.com/doc/v8-stable/configuration/templates.html
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Reply via email to