Re: [rsyslog] Question on multiline log messages

2018-09-06 Thread David Lang
There is an option for imfile to split a file into multiline messages based on a regex, that is probably the right starting point. Note that a LOT of log processing tools assume a log message is a single line, so you probably want to have newlines escaped in the message before sending it to

[rsyslog] filter rules

2018-09-06 Thread Adam Barnett via rsyslog
Hi All I am having an issue with Rsyslog and it driving my up the wall. I have a few hosts that don't send logging that is correctly formatted ( it changes depending on the error they generate , sigh ) I have the following config /etc/rsyslog.conf: $MaxMessageSize 32k $ModLoad imuxsock.so #

[rsyslog] rSyslog separation and forwarding

2018-09-06 Thread sarjit yadav via rsyslog
Hi Experts, I am working on implementation of syslog server and running rsyslog sw version="8.24.0 on CentOS. Currently all logs from client are written in /var/log/message file and looking suggestions to achieve - - ONLY write logs from specific host to dedicated partition

Re: [rsyslog] Parsed data not loading to database all zeros

2018-09-06 Thread Jason Prouty
My Log files are empty From: rsyslog on behalf of David Lang Sent: Wednesday, September 5, 2018 5:34 PM To: rsyslog-users Subject: Re: [rsyslog] Parsed data not loading to database all zeros write the data to a file with the template that you are using to

Re: [rsyslog] filter rules

2018-09-06 Thread David Lang
First off, different files don't isolate processing, the include makes it as if you did a cut-n-paste of the contents of the file into the main rsyslog.conf file at that point, so all logs are going to hit all rules in all include files when you have bad messages, log them to a file with the

Re: [rsyslog] Question on multiline log messages

2018-09-06 Thread Rich Megginson via rsyslog
On 09/06/2018 11:30 AM, David Lang wrote: On Thu, 6 Sep 2018, Rich Megginson wrote: On 09/06/2018 12:00 PM, Noriko Hosoi via rsyslog wrote: Thank you for your response, David. On 09/06/2018 06:18 AM, David Lang wrote: There is an option for imfile to split a file into multiline messages

Re: [rsyslog] filter rules

2018-09-06 Thread David Lang
but nothing resets the ruleset after that point, so everything after the 'include' would be part of the server ruleset, right? (another reason to use the current syntax) David Lang On Thu, 6 Sep 2018, Rainer Gerhards wrote: Date: Thu, 6 Sep 2018 20:33:50 +0200 From: Rainer Gerhards

Re: [rsyslog] Question on multiline log messages

2018-09-06 Thread David Lang
On Thu, 6 Sep 2018, Rich Megginson wrote: That is, if multiline specifically means records like this: first line of recordA    recordA second line    recordA third line first line of recordB Perhaps Noriko and I mean something slightly different by "multiline" - a single message spread

Re: [rsyslog] rsyslog 7.2.6 option to close tcp/udp sockets after mainqueue backlog

2018-09-06 Thread Prasad Koya via rsyslog
Thank you. We are looking at configuration options in https://www.rsyslog.com/doc/v7-stable/configuration/action/index.html Is there any page with bit more description of above configuration options. For example $ActionQueueSize Is above ActionQueueSize different from MainQ size? If this

Re: [rsyslog] Question on multiline log messages

2018-09-06 Thread Rich Megginson via rsyslog
On 09/06/2018 12:00 PM, Noriko Hosoi via rsyslog wrote: Thank you for your response, David. On 09/06/2018 06:18 AM, David Lang wrote: There is an option for imfile to split a file into multiline messages based on a regex, that is probably the right starting point. Yes, we also expected

Re: [rsyslog] filter rules

2018-09-06 Thread Rainer Gerhards
If I see correctly, the inputs are bound to ruleset server, which seems to be in 98-... So the other rules are not hit. Side note: current config format would greatly enhance readability and reduce error probability. Rainer Sent from phone, thus brief. David Lang schrieb am Do., 6. Sep.

Re: [rsyslog] Question on multiline log messages

2018-09-06 Thread Noriko Hosoi via rsyslog
Thank you for your response, David. On 09/06/2018 06:18 AM, David Lang wrote: There is an option for imfile to split a file into multiline messages based on a regex, that is probably the right starting point. Yes, we also expected startmsg.regex could be used for our purpose.

Re: [rsyslog] rSyslog separation and forwarding

2018-09-06 Thread David Lang
On Thu, 6 Sep 2018, sarjit yadav via rsyslog wrote: I am working on implementation of syslog server and running rsyslog sw version="8.24.0 on CentOS. Currently all logs from client are written in /var/log/message file and looking suggestions to achieve - - ONLY write logs from specific host to

Re: [rsyslog] Question on multiline log messages

2018-09-06 Thread David Lang
On Thu, 6 Sep 2018, Rich Megginson wrote: On 09/06/2018 12:00 PM, Noriko Hosoi via rsyslog wrote: Thank you for your response, David. On 09/06/2018 06:18 AM, David Lang wrote: There is an option for imfile to split a file into multiline messages based on a regex, that is probably the right

Re: [rsyslog] filter rules

2018-09-06 Thread Rainer Gerhards
David Lang schrieb am Do., 6. Sep. 2018, 20:38: > but nothing resets the ruleset after that point, so everything after the > 'include' would be part of the server ruleset, right? > Yes (of course, lol) > (another reason to use the current syntax) > Actually this was one of the two use cases

Re: [rsyslog] Question on multiline log messages

2018-09-06 Thread Rich Megginson via rsyslog
On 09/06/2018 11:57 AM, David Lang wrote: On Thu, 6 Sep 2018, Rich Megginson wrote: That is, if multiline specifically means records like this: first line of recordA    recordA second line    recordA third line first line of recordB Perhaps Noriko and I mean something slightly different

Re: [rsyslog] Question on multiline log messages

2018-09-06 Thread Rich Megginson via rsyslog
On 09/06/2018 12:49 PM, Rich Megginson wrote: On 09/06/2018 11:30 AM, David Lang wrote: On Thu, 6 Sep 2018, Rich Megginson wrote: On 09/06/2018 12:00 PM, Noriko Hosoi via rsyslog wrote: Thank you for your response, David. On 09/06/2018 06:18 AM, David Lang wrote: There is an option for

Re: [rsyslog] Question on multiline log messages

2018-09-06 Thread Rich Megginson via rsyslog
On 09/06/2018 03:45 PM, David Lang wrote: On Thu, 6 Sep 2018, Rich Megginson wrote: Just to clarify - rsyslog does not currently support "multiline" docker json-file nor crio logs because there is no endmsg.regex - we would need to add support for that first in order to use mmnormalize repeat

Re: [rsyslog] Question on multiline log messages

2018-09-06 Thread David Lang
On Thu, 6 Sep 2018, Rich Megginson wrote: Just to clarify - rsyslog does not currently support "multiline" docker json-file nor crio logs because there is no endmsg.regex - we would need to add support for that first in order to use mmnormalize repeat and foreach as you have described above.