Re: [rsyslog] How can I use high precision forwarding format, but still preserve the Postfix process name in forwarded messages?

[deoren]

On 8/8/2017 1:10 PM, rsyslog-users-lists.adiscon@whyaskwhy.org wrote:

On 8/8/17 2:30 AM, Rainer Gerhards wrote:

>>
Check what APP-NAME, PROCID and MSGID contain, which are derived from 
the tag.


RFC5424 tells you where these parts are to be placed in the header.



It appears that this lack of a colon is confusing pflogsumm when the 
daily
cron job calls this script to generate a daily report of the mail 
activity

recorded on our central rsyslog instance.


that would indicated that pflogsumm does not properly handle RFC5424 
message.


HTH
Rainer


Thank you for your feedback, I appreciate you taking the time to respond.

When I enable debug logging I see that the colon is nowhere to be seen 
in 'programname' or 'APP-NAME' when in any of the forwarding formats 
(which I understand to be the norm), but is present in the syslogtag 
property for Traditional and Forward formats, not present for the 
Protocol23 forwarding format.



# RSYSLOG_TraditionalForwardFormat:
syslogtag 'postfix/qmgr[29132]:', programname: 'postfix/qmgr', APP-NAME: 
'postfix/qmgr', PROCID: '29132', MSGID: '-',



# RSYSLOG_ForwardFormat:
syslogtag 'postfix/qmgr[29132]:', programname: 'postfix/qmgr', APP-NAME: 
'postfix/qmgr', PROCID: '29132', MSGID: '-',



# RSYSLOG_SyslogProtocol23Format:
syslogtag 'postfix/qmgr[29132]', programname: 'postfix/qmgr', APP-NAME: 
'postfix/qmgr', PROCID: '29132', MSGID: '-',


When rsyslog saves a stream of Protocol23 formatted messages to disk, I 
assumed that the RSYSLOG_FileFormat template would source the syslogtag 
property and save that entire value to disk as-is. Does something else 
happen instead?


If I can provide further information, please let me know.

Thanks.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] How can I use high precision forwarding format, but still preserve the Postfix process name in forwarded messages?

[rsyslog-users-lists . adiscon . net]

On 8/8/17 2:30 AM, Rainer Gerhards wrote:

2017-08-08 6:36 GMT+02:00 deoren



Does the 'RSYSLOG_SyslogProtocol23Format' format intentionally drop colons
from the 'syslogtag' property?


Well, this format is RFC5424, and RFC5424 does not have syslogtag as
you know it. See RFC5424 Sect A.1 for the relationship. This is the
relevant quote for your question:

-

   The MSG part of the message is described as TAG and CONTENT in RFC
3164.  In this document, MSG is what was called CONTENT in RFC 3164.
The TAG is now part of the header, but not as a single field.  The
TAG has been split into APP-NAME, PROCID, and MSGID.  This does not
totally resemble the usage of TAG, but provides the same
functionality for most of the cases.

-

I have not actually checked the code, but I think we drop the colon as
part of this transformation process.



On the original system I use the RSYSLOG_DebugFormat template and I see that
'syslogtag' contains a value like this (note the colon):

'postfix/qmgr[1144]:'

but when forwarded, the RSYSLOG_DebugFormat template shows the syslogtag as
containing (note the lack of a colon):

'postfix/qmgr[1144]'


Check what APP-NAME, PROCID and MSGID contain, which are derived from the tag.

RFC5424 tells you where these parts are to be placed in the header.



It appears that this lack of a colon is confusing pflogsumm when the daily
cron job calls this script to generate a daily report of the mail activity
recorded on our central rsyslog instance.


that would indicated that pflogsumm does not properly handle RFC5424 message.

HTH
Rainer


Thank you for your feedback, I appreciate you taking the time to respond.

When I enable debug logging I see that the colon is nowhere to be seen 
in 'programname' or 'APP-NAME' when in any of the forwarding formats 
(which I understand to be the norm), but is present in the syslogtag 
property for Traditional and Forward formats, not present for the 
Protocol23 forwarding format.



# RSYSLOG_TraditionalForwardFormat:
syslogtag 'postfix/qmgr[29132]:', programname: 'postfix/qmgr', APP-NAME: 
'postfix/qmgr', PROCID: '29132', MSGID: '-',



# RSYSLOG_ForwardFormat:
syslogtag 'postfix/qmgr[29132]:', programname: 'postfix/qmgr', APP-NAME: 
'postfix/qmgr', PROCID: '29132', MSGID: '-',



# RSYSLOG_SyslogProtocol23Format:
syslogtag 'postfix/qmgr[29132]', programname: 'postfix/qmgr', APP-NAME: 
'postfix/qmgr', PROCID: '29132', MSGID: '-',


When rsyslog saves a stream of Protocol23 formatted messages to disk, I 
assumed that the RSYSLOG_FileFormat template would source the syslogtag 
property and save that entire value to disk as-is. Does something else 
happen instead?


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] How can I use high precision forwarding format, but still preserve the Postfix process name in forwarded messages?

[Rainer Gerhards]

2017-08-08 6:36 GMT+02:00 deoren
:
> On 8/5/17 11:42 PM, deoren wrote:
>>
>> On 8/5/17 10:59 PM, deoren wrote:
>>>
>>> I've recently converted all of our nodes from forwarding messages from
>>> the default forwarding format to using the 'RSYSLOG_SyslogProtocol23Format'
>>> format.
>>>
>>> I only did light research beforehand (so I can only blame myself), but
>>> when our relay nodes log in either 'RSYSLOG_TraditionalFileFormat' or
>>> 'RSYSLOG_FileFormat' the process name is recorded in the log file(s). When
>>> we forward messages using the default forwarding format, that information
>>> seems to come across as expected. When our client nodes forward using the
>>> 'RSYSLOG_SyslogProtocol23Format' format the process name information is
>>> "lost" and not recorded within the local log files on our receiver.
>>>
>>> DETAILS
>>>
>>> sender:
>>>
>>> * rsyslog 8.28.0
>>> * Ubuntu 16.04
>>> * using official PPA
>>> * Postfix mail relay node
>>> * sending in 'RSYSLOG_SyslogProtocol23Format' format via RELP
>>> * recording locally in 'RSYSLOG_FileFormat' format
>>> * local messages appear to retain 'postfix/smtpd[]' format (which is
>>> desired)
>>>
>>> receiver:
>>>
>>> * rsyslog 8.28.0
>>> * Ubuntu 16.04
>>> * using official PPA
>>> * receiving via RELP
>>> * recording locally in 'RSYSLOG_FileFormat' format
>>> * messages saved appear to no longer retain the process name (e.g.,
>>> 'postfix[]')
>>>
>>>
>>> Turning to Google, I found this[1] bug report which explained the problem
>>> much better than I ever could. I later found another thread[2] where the
>>> problem was discussed, but no resolution reached. The final post to that
>>> thread[4] illustrated that the syslogtag value is coming across with the
>>> values needed in the syslogtag field, but both programname and app-name
>>> properties were missing the Postfix component/process name information.
>>>
>>> Is there a way to subsclass or inherit the RSYSLOG_FileFormat template
>>> and override the value it uses for app-name? I don't know what I am doing in
>>> that regarding, but I'm willing to learn. I'd really like to preserve the
>>> Postfix process name in the log messages saved on our receiver node.
>>>
>>> In absence of the understanding necessary to override the existing
>>> template, one workaround I'm considering is having our client nodes check
>>> the facility before forwarding messages on to the central receiver. If the
>>> facility is 'mail', send using traditional forwarding format, otherwise send
>>> everything else using the newer forwarding format. I'd rather have precision
>>> timestamps on both, but having the process name available in the mail logs
>>> is more important in our use case.
>>>
>>> Thank you in advance for your help!
>>>
>>>
>>> References:
>>>
>>> [1] https://github.com/rsyslog/rsyslog/issues/168
>>>
>>> [2] http://kb.monitorware.com/app-name-programname-t12945.html
>>>
>>> [3] http://kb.monitorware.com/post27055.html#p27055
>>>
>>> [4] http://kb.monitorware.com/post27061.html#p27061
>>>
>>> [5] http://www.rsyslog.com/doc/v8-stable/configuration/templates.html
>>
>>
>> After hitting Send, I found the 'parser.permitSlashInProgramName' option
>> listed on the properties page which seems to do what I'm looking for. I
>> enable it on the client nodes and the receiver seems to pick it up without
>> requiring that I also enable it there (but of course I will for
>> consistency).
>>
>> I submitted a bug report against the rsyslog-doc repo to include the
>> global option on the global documentation page.
>>
>> https://github.com/rsyslog/rsyslog-doc/issues/359
>>
>> According to the notes for that option, it appears it was added in a
>> fairly recent release. Many thanks to the devs for including it!
>
>
> Does the 'RSYSLOG_SyslogProtocol23Format' format intentionally drop colons
> from the 'syslogtag' property?

Well, this format is RFC5424, and RFC5424 does not have syslogtag as
you know it. See RFC5424 Sect A.1 for the relationship. This is the
relevant quote for your question:

-

  The MSG part of the message is described as TAG and CONTENT in RFC
   3164.  In this document, MSG is what was called CONTENT in RFC 3164.
   The TAG is now part of the header, but not as a single field.  The
   TAG has been split into APP-NAME, PROCID, and MSGID.  This does not
   totally resemble the usage of TAG, but provides the same
   functionality for most of the cases.

-

I have not actually checked the code, but I think we drop the colon as
part of this transformation process.

>
> On the original system I use the RSYSLOG_DebugFormat template and I see that
> 'syslogtag' contains a value like this (note the colon):
>
> 'postfix/qmgr[1144]:'
>
> but when forwarded, the RSYSLOG_DebugFormat template shows the syslogtag as
> containing (note the lack of a colon):
>
> 'postfix/qmgr[1144]'

Check what APP-NAME, PROCID and MSGID contain, 

Re: [rsyslog] How can I use high precision forwarding format, but still preserve the Postfix process name in forwarded messages?

[deoren]

On 8/5/17 11:42 PM, deoren wrote:

On 8/5/17 10:59 PM, deoren wrote:
I've recently converted all of our nodes from forwarding messages from 
the default forwarding format to using the 
'RSYSLOG_SyslogProtocol23Format' format.


I only did light research beforehand (so I can only blame myself), but 
when our relay nodes log in either 'RSYSLOG_TraditionalFileFormat' or 
'RSYSLOG_FileFormat' the process name is recorded in the log file(s). 
When we forward messages using the default forwarding format, that 
information seems to come across as expected. When our client nodes 
forward using the 'RSYSLOG_SyslogProtocol23Format' format the process 
name information is "lost" and not recorded within the local log files 
on our receiver.


DETAILS

sender:

* rsyslog 8.28.0
* Ubuntu 16.04
* using official PPA
* Postfix mail relay node
* sending in 'RSYSLOG_SyslogProtocol23Format' format via RELP
* recording locally in 'RSYSLOG_FileFormat' format
* local messages appear to retain 'postfix/smtpd[]' format (which 
is desired)


receiver:

* rsyslog 8.28.0
* Ubuntu 16.04
* using official PPA
* receiving via RELP
* recording locally in 'RSYSLOG_FileFormat' format
* messages saved appear to no longer retain the process name (e.g., 
'postfix[]')



Turning to Google, I found this[1] bug report which explained the 
problem much better than I ever could. I later found another thread[2] 
where the problem was discussed, but no resolution reached. The final 
post to that thread[4] illustrated that the syslogtag value is coming 
across with the values needed in the syslogtag field, but both 
programname and app-name properties were missing the Postfix 
component/process name information.


Is there a way to subsclass or inherit the RSYSLOG_FileFormat template 
and override the value it uses for app-name? I don't know what I am 
doing in that regarding, but I'm willing to learn. I'd really like to 
preserve the Postfix process name in the log messages saved on our 
receiver node.


In absence of the understanding necessary to override the existing 
template, one workaround I'm considering is having our client nodes 
check the facility before forwarding messages on to the central 
receiver. If the facility is 'mail', send using traditional forwarding 
format, otherwise send everything else using the newer forwarding 
format. I'd rather have precision timestamps on both, but having the 
process name available in the mail logs is more important in our use 
case.


Thank you in advance for your help!


References:

[1] https://github.com/rsyslog/rsyslog/issues/168

[2] http://kb.monitorware.com/app-name-programname-t12945.html

[3] http://kb.monitorware.com/post27055.html#p27055

[4] http://kb.monitorware.com/post27061.html#p27061

[5] http://www.rsyslog.com/doc/v8-stable/configuration/templates.html


After hitting Send, I found the 'parser.permitSlashInProgramName' option 
listed on the properties page which seems to do what I'm looking for. I 
enable it on the client nodes and the receiver seems to pick it up 
without requiring that I also enable it there (but of course I will for 
consistency).


I submitted a bug report against the rsyslog-doc repo to include the 
global option on the global documentation page.


https://github.com/rsyslog/rsyslog-doc/issues/359

According to the notes for that option, it appears it was added in a 
fairly recent release. Many thanks to the devs for including it!


Does the 'RSYSLOG_SyslogProtocol23Format' format intentionally drop 
colons from the 'syslogtag' property?


On the original system I use the RSYSLOG_DebugFormat template and I see 
that 'syslogtag' contains a value like this (note the colon):


'postfix/qmgr[1144]:'

but when forwarded, the RSYSLOG_DebugFormat template shows the syslogtag 
as containing (note the lack of a colon):


'postfix/qmgr[1144]'

It appears that this lack of a colon is confusing pflogsumm when the 
daily cron job calls this script to generate a daily report of the mail 
activity recorded on our central rsyslog instance.


Thank you for your help.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] How can I use high precision forwarding format, but still preserve the Postfix process name in forwarded messages?

[David Lang]

I'll post something in more detail later.

On Sat, 5 Aug 2017, deoren wrote:

That said, thank you for the tips. I know I'm eventually going to have to 
look at using JSON since most of the popular tool chains I'm researching seem 
to prefer it (e.g., Elastic Stack, Graylog), so I'm definitely interested in 
learning more about that approach.


Are there any good guides that cover the steps you mention in detail? I 
understand the basic idea of what you're suggesting, but perhaps not the step 
by step implementation details.

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] How can I use high precision forwarding format, but still preserve the Postfix process name in forwarded messages?

[deoren]

On 8/5/17 11:28 PM, David Lang wrote:
on the receiver, write a log with the format rawmsg or use the 
RSYSLOG_DebugFormat and look at the rawmsg line there. Let's see exactly 
what is being sent to see if the data is being lost at transmit or on 
reception.


Personally, I have my senders reformat the data so that the body of the 
message is in JSON, and then I create a $!trusted variable that I have 
contain various metadata. each machine that processes the message adds 
to the metadata (when it was received, what machine recived it, what IP 
it was recevied from,etc)


This is a good place to put high res timing info or syslogtag info to 
make sure that nothing mangles it in processing. On the final receiver, 
you then parse the JSON and have access to everything you every want.


This is an indirect way of solving the problem you have, but it solves a 
bunch of other problems at the same time, and doesn't require the 
SyslogProtocol123Format to get the timestamp you want (you may even just 
include the timestamp in various formats so you don't have to mess with 
changing it's format later)


David Lang


I responded back to my original email a few minutes ago before seeing 
your reply, but it looks like setting this global option on the clients 
will do what I need:


global (
parser.permitSlashInProgramName="on"
)

That said, thank you for the tips. I know I'm eventually going to have 
to look at using JSON since most of the popular tool chains I'm 
researching seem to prefer it (e.g., Elastic Stack, Graylog), so I'm 
definitely interested in learning more about that approach.


Are there any good guides that cover the steps you mention in detail? I 
understand the basic idea of what you're suggesting, but perhaps not the 
step by step implementation details.


Thank you for your help.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] How can I use high precision forwarding format, but still preserve the Postfix process name in forwarded messages?

[deoren]

On 8/5/17 10:59 PM, deoren wrote:
I've recently converted all of our nodes from forwarding messages from 
the default forwarding format to using the 
'RSYSLOG_SyslogProtocol23Format' format.


I only did light research beforehand (so I can only blame myself), but 
when our relay nodes log in either 'RSYSLOG_TraditionalFileFormat' or 
'RSYSLOG_FileFormat' the process name is recorded in the log file(s). 
When we forward messages using the default forwarding format, that 
information seems to come across as expected. When our client nodes 
forward using the 'RSYSLOG_SyslogProtocol23Format' format the process 
name information is "lost" and not recorded within the local log files 
on our receiver.


DETAILS

sender:

* rsyslog 8.28.0
* Ubuntu 16.04
* using official PPA
* Postfix mail relay node
* sending in 'RSYSLOG_SyslogProtocol23Format' format via RELP
* recording locally in 'RSYSLOG_FileFormat' format
* local messages appear to retain 'postfix/smtpd[]' format (which is 
desired)


receiver:

* rsyslog 8.28.0
* Ubuntu 16.04
* using official PPA
* receiving via RELP
* recording locally in 'RSYSLOG_FileFormat' format
* messages saved appear to no longer retain the process name (e.g., 
'postfix[]')



Turning to Google, I found this[1] bug report which explained the 
problem much better than I ever could. I later found another thread[2] 
where the problem was discussed, but no resolution reached. The final 
post to that thread[4] illustrated that the syslogtag value is coming 
across with the values needed in the syslogtag field, but both 
programname and app-name properties were missing the Postfix 
component/process name information.


Is there a way to subsclass or inherit the RSYSLOG_FileFormat template 
and override the value it uses for app-name? I don't know what I am 
doing in that regarding, but I'm willing to learn. I'd really like to 
preserve the Postfix process name in the log messages saved on our 
receiver node.


In absence of the understanding necessary to override the existing 
template, one workaround I'm considering is having our client nodes 
check the facility before forwarding messages on to the central 
receiver. If the facility is 'mail', send using traditional forwarding 
format, otherwise send everything else using the newer forwarding 
format. I'd rather have precision timestamps on both, but having the 
process name available in the mail logs is more important in our use case.


Thank you in advance for your help!


References:

[1] https://github.com/rsyslog/rsyslog/issues/168

[2] http://kb.monitorware.com/app-name-programname-t12945.html

[3] http://kb.monitorware.com/post27055.html#p27055

[4] http://kb.monitorware.com/post27061.html#p27061

[5] http://www.rsyslog.com/doc/v8-stable/configuration/templates.html


After hitting Send, I found the 'parser.permitSlashInProgramName' option 
listed on the properties page which seems to do what I'm looking for. I 
enable it on the client nodes and the receiver seems to pick it up 
without requiring that I also enable it there (but of course I will for 
consistency).


I submitted a bug report against the rsyslog-doc repo to include the 
global option on the global documentation page.


https://github.com/rsyslog/rsyslog-doc/issues/359

According to the notes for that option, it appears it was added in a 
fairly recent release. Many thanks to the devs for including it!

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] How can I use high precision forwarding format, but still preserve the Postfix process name in forwarded messages?

[David Lang]

on the receiver, write a log with the format rawmsg or use the 
RSYSLOG_DebugFormat and look at the rawmsg line there. Let's see exactly what is 
being sent to see if the data is being lost at transmit or on reception.


Personally, I have my senders reformat the data so that the body of the message 
is in JSON, and then I create a $!trusted variable that I have contain various 
metadata. each machine that processes the message adds to the metadata (when it 
was received, what machine recived it, what IP it was recevied from,etc)


This is a good place to put high res timing info or syslogtag info to make sure 
that nothing mangles it in processing. On the final receiver, you then parse the 
JSON and have access to everything you every want.


This is an indirect way of solving the problem you have, but it solves a bunch 
of other problems at the same time, and doesn't require the 
SyslogProtocol123Format to get the timestamp you want (you may even just 
include the timestamp in various formats so you don't have to mess with changing 
it's format later)


David Lang

 On Sat, 5 Aug 2017, deoren wrote:


Date: Sat, 5 Aug 2017 22:59:59 -0500
From: deoren 
Reply-To: rsyslog-users 
To: rsyslog@lists.adiscon.com
Subject: [rsyslog] How can I use high precision forwarding format,
but still preserve the Postfix process name in forwarded messages?

I've recently converted all of our nodes from forwarding messages from the 
default forwarding format to using the 'RSYSLOG_SyslogProtocol23Format' 
format.


I only did light research beforehand (so I can only blame myself), but when 
our relay nodes log in either 'RSYSLOG_TraditionalFileFormat' or 
'RSYSLOG_FileFormat' the process name is recorded in the log file(s). When we 
forward messages using the default forwarding format, that information seems 
to come across as expected. When our client nodes forward using the 
'RSYSLOG_SyslogProtocol23Format' format the process name information is 
"lost" and not recorded within the local log files on our receiver.


DETAILS

sender:

* rsyslog 8.28.0
* Ubuntu 16.04
* using official PPA
* Postfix mail relay node
* sending in 'RSYSLOG_SyslogProtocol23Format' format via RELP
* recording locally in 'RSYSLOG_FileFormat' format
* local messages appear to retain 'postfix/smtpd[]' format (which is 
desired)


receiver:

* rsyslog 8.28.0
* Ubuntu 16.04
* using official PPA
* receiving via RELP
* recording locally in 'RSYSLOG_FileFormat' format
* messages saved appear to no longer retain the process name (e.g., 
'postfix[]')



Turning to Google, I found this[1] bug report which explained the problem 
much better than I ever could. I later found another thread[2] where the 
problem was discussed, but no resolution reached. The final post to that 
thread[4] illustrated that the syslogtag value is coming across with the 
values needed in the syslogtag field, but both programname and app-name 
properties were missing the Postfix component/process name information.


Is there a way to subsclass or inherit the RSYSLOG_FileFormat template and 
override the value it uses for app-name? I don't know what I am doing in that 
regarding, but I'm willing to learn. I'd really like to preserve the Postfix 
process name in the log messages saved on our receiver node.


In absence of the understanding necessary to override the existing template, 
one workaround I'm considering is having our client nodes check the facility 
before forwarding messages on to the central receiver. If the facility is 
'mail', send using traditional forwarding format, otherwise send everything 
else using the newer forwarding format. I'd rather have precision timestamps 
on both, but having the process name available in the mail logs is more 
important in our use case.


Thank you in advance for your help!


References:

[1] https://github.com/rsyslog/rsyslog/issues/168

[2] http://kb.monitorware.com/app-name-programname-t12945.html

[3] http://kb.monitorware.com/post27055.html#p27055

[4] http://kb.monitorware.com/post27061.html#p27061

[5] http://www.rsyslog.com/doc/v8-stable/configuration/templates.html
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
LIKE THAT.



___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] How can I use high precision forwarding format, but still preserve the Postfix process name in forwarded messages?

[deoren]

I've recently converted all of our nodes from forwarding messages from 
the default forwarding format to using the 
'RSYSLOG_SyslogProtocol23Format' format.


I only did light research beforehand (so I can only blame myself), but 
when our relay nodes log in either 'RSYSLOG_TraditionalFileFormat' or 
'RSYSLOG_FileFormat' the process name is recorded in the log file(s). 
When we forward messages using the default forwarding format, that 
information seems to come across as expected. When our client nodes 
forward using the 'RSYSLOG_SyslogProtocol23Format' format the process 
name information is "lost" and not recorded within the local log files 
on our receiver.


DETAILS

sender:

* rsyslog 8.28.0
* Ubuntu 16.04
* using official PPA
* Postfix mail relay node
* sending in 'RSYSLOG_SyslogProtocol23Format' format via RELP
* recording locally in 'RSYSLOG_FileFormat' format
* local messages appear to retain 'postfix/smtpd[]' format (which is 
desired)


receiver:

* rsyslog 8.28.0
* Ubuntu 16.04
* using official PPA
* receiving via RELP
* recording locally in 'RSYSLOG_FileFormat' format
* messages saved appear to no longer retain the process name (e.g., 
'postfix[]')



Turning to Google, I found this[1] bug report which explained the 
problem much better than I ever could. I later found another thread[2] 
where the problem was discussed, but no resolution reached. The final 
post to that thread[4] illustrated that the syslogtag value is coming 
across with the values needed in the syslogtag field, but both 
programname and app-name properties were missing the Postfix 
component/process name information.


Is there a way to subsclass or inherit the RSYSLOG_FileFormat template 
and override the value it uses for app-name? I don't know what I am 
doing in that regarding, but I'm willing to learn. I'd really like to 
preserve the Postfix process name in the log messages saved on our 
receiver node.


In absence of the understanding necessary to override the existing 
template, one workaround I'm considering is having our client nodes 
check the facility before forwarding messages on to the central 
receiver. If the facility is 'mail', send using traditional forwarding 
format, otherwise send everything else using the newer forwarding 
format. I'd rather have precision timestamps on both, but having the 
process name available in the mail logs is more important in our use case.


Thank you in advance for your help!


References:

[1] https://github.com/rsyslog/rsyslog/issues/168

[2] http://kb.monitorware.com/app-name-programname-t12945.html

[3] http://kb.monitorware.com/post27055.html#p27055

[4] http://kb.monitorware.com/post27061.html#p27061

[5] http://www.rsyslog.com/doc/v8-stable/configuration/templates.html
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.