Hi,
I've made the changes you recommended (turning off message reduction, removing
the trailing /, and took out more comments.
Your comment, " remember, it doesn't stop processing the log at the first
match, it keeps going in case there are other matches as well."
Really? I thought the " stop }" at the end of a specific config was telling
rsyslog to stop processing the message and/or looking for other matches?
If rsyslog is still looking for other matches then why doesn't every message
end up in my "Unclassified" folder since I'm matching against "*.*" for my
Unknowns filter?
Is there a right/better way to have unmatched messages be dealt with? The next
step in our chain is Splunk, and it is setup to monitor specific folders for a
given "source/source-type" I need some way to guarantee that messages that
match "fromhost-ip" or "startswith" only end up in a specific folder. And all
other messages that do not match either of those go into another folder that I
can monitor and build new filters for or modify existing filters (example,
adding another ESXi host to the list).
Here is my current rsyslog.conf, the extra ESXi.conf, and an example output
from the msgdebug.log of a message that ended up in the Unclassified folder:
####MAIN CONFIG FILE:
root@whqlrsyslog02 # cat /etc/rsyslog.conf
$umask 0000
### for debug
*.* /var/splunk-syslog/msgdebug.log;RSYSLOG_DebugFormat
$ModLoad immark.so
$MarkMessagePeriod 3600
$ModLoad imuxsock.so
$RepeatedMsgReduction off
$ModLoad imklog.so
$klogConsoleLogLevel 1
$IncludeConfig /etc/rsyslog.d/*.conf
$IncludeConfig /etc/rsyslog.d/*.template
# Unknowns
$umask 0000
template(name="Unclassified" type="string"
string="/var/splunk-syslog/Unclassified/%HOSTNAME%/%FROMHOST%-%$NOW%.log")
*.* action(type="omfile" dirCreateMode="0755" FileCreateMode="0644"
dynaFile="Unclassified")
if (
($syslogfacility-text == 'kern') and
($syslogseverity <= 4 ) and not
($msg contains 'IN=' and $msg contains 'OUT=')
) or (
($syslogseverity <= 3 ) and not
($syslogfacility-text == 'authpriv')
)
then {
/dev/tty10
|/dev/xconsole
}
*.emerg :omusrmsg:*
if ($syslogfacility-text == 'kern') and
($msg contains 'IN=' and $msg contains 'OUT=')
then {
/var/log/firewall
stop
}
if ($programname == 'acpid' or $syslogtag == '[acpid]:') and
($syslogseverity <= 5 )
then {
/var/log/acpid
stop
}
if ($programname == 'NetworkManager') or
($programname startswith 'nm-')
then {
/var/log/NetworkManager
stop
}
mail.* /var/log/mail
mail.info /var/log/mail.info
mail.warning /var/log/mail.warn
mail.err /var/log/mail.err
news.crit /var/log/news/news.crit
news.err /var/log/news/news.err
news.notice /var/log/news/news.notice
*.=warning;*.=err /var/log/warn
*.crit /var/log/warn
*.*;mail.none;news.none /var/log/messages
local0.*;local1.* /var/log/localmessages
local2.*;local3.* /var/log/localmessages
local4.*;local5.* /var/log/localmessages
local6.*;local7.* /var/log/localmessages
###
####FROM EXTRA CONFIG FILE:
root@whqlrsyslog02 # cat /etc/rsyslog.d/ESXi.conf
template(name="ESXi_app" type="string"
string="/var/splunk-syslog/ESXi/%FROMHOST%/%FROMHOST%-%$NOW%.log")
if $fromhost-ip == ["100.31.20.101",
"100.31.20.102",
"100.200.20.231",
"100.200.20.232"]
then {
action(type="omfile" dirCreateMode="0755" FileCreateMode="0644"
dynaFile="ESXi_app")
stop }
####FROM THE DEBUG FILE:
Debug line with all properties:
FROMHOST: 'sdcurpe02.mycompany.com', fromhost-ip: '100.200.20.232', HOSTNAME:
'sdcurpe02.mycompany.com', PRI: 14,
syslogtag 'storageRM:', programname: 'storageRM', APP-NAME: 'storageRM',
PROCID: '-', MSGID: '-',
TIMESTAMP: 'Dec 5 13:50:52', STRUCTURED-DATA: '-',
msg: ' <EL-SDC2003C, 0> Some host is down, need to reset the slot allocation'
escaped msg: ' <EL-SDC2003C, 0> Some host is down, need to reset the slot
allocation'
inputname: imtcp rawmsg: '<14>2016-12-05T13:50:52.367Z sdcurpe02.mycompany.com
storageRM: <EL-SDC2003C, 0> Some host is down, need to reset the slot
allocation'
Debug line with all properties:
FROMHOST: 'sdcurpe01.mycompany.com', fromhost-ip: '100.200.20.231', HOSTNAME:
'sdcurpe01.mycompany.com', PRI: 14,
syslogtag 'storageRM:', programname: 'storageRM', APP-NAME: 'storageRM',
PROCID: '-', MSGID: '-',
TIMESTAMP: 'Dec 5 13:42:37', STRUCTURED-DATA: '-',
msg: ' <EL-SDC2003C, 0> Number of hosts has changed to 6'
escaped msg: ' <EL-SDC2003C, 0> Number of hosts has changed to 6'
inputname: imtcp rawmsg: '<14>2016-12-05T13:42:37.851Z sdcurpe01.mycompany.com
storageRM: <EL-SDC2003C, 0> Number of hosts has changed to 6'
Again, thank you for your patience and support!
Patrick H Swartz
-----Original Message-----
From: rsyslog-boun...@lists.adiscon.com
[mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of David Lang
Sent: Friday, December 02, 2016 4:35 PM
To: rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] filters question
On Fri, 2 Dec 2016, Swartz, Patrick wrote:
> root@whqlrsyslog01 # rsyslogd -N1
> rsyslogd: version 8.4.0, config validation run (level 1), master
> config /etc/rsyslog.conf
> rsyslogd: End of config validation run. Bye.
>
> Here is my full rsyslogd.conf (minus comments). My additions/changes are
> marked to the side (the comments are not in the file), everything else is
> stock from SUSE.
>
> root@whqlrsyslog01 # sed -e '/\s*#.*$/d' -e '/^\s*$/d' /etc/rsyslog.conf
> $umask 0000 ######### Added by pswartz
> *.* /var/splunk-syslog/msgdebug.log;RSYSLOG_DebugFormat
> ######### Added by pswartz
> $ModLoad immark.so
> $MarkMessagePeriod 3600
> $ModLoad imuxsock.so
> $RepeatedMsgReduction on
we actually recommend not doing message reduction, it's a lot easier for
monitoring to deal with the same log message 500 times than a log message that
says 'last message repeated 499 times'
> $ModLoad imklog.so
> $klogConsoleLogLevel 1
> $IncludeConfig /etc/rsyslog.d/*.conf ######### Added by
> pswartz
> $IncludeConfig /etc/rsyslog.d/*.template ######### Added by
> pswartz
> $umask 0000
> template(name="Unclassified" type="string"
> string="/var/splunk-syslog/Unclassified/%HOSTNAME%/%FROMHOST%-%$NOW%.log")
> ##### Added by pswartz
> *.* action(type="omfile" dirCreateMode="0755" FileCreateMode="0644"
> dynaFile="Unclassified") ######### Added by pswartz
> if ( \
are these trailing slashes in the file? they should not be needed
> /* kernel up to warning except of firewall */ \
I hope this is one of the comments you say are not actually in the file.
> ($syslogfacility-text == 'kern') and \
> ($syslogseverity <= 4 /* warning */ ) and not \
> ($msg contains 'IN=' and $msg contains 'OUT=') \
> ) or ( \
> /* up to errors except of facility authpriv */ \
> ($syslogseverity <= 3 /* errors */ ) and not \
> ($syslogfacility-text == 'authpriv') \
> ) \
> then {
> /dev/tty10
> |/dev/xconsole
> }
> *.emerg :omusrmsg:*
> if ($syslogfacility-text == 'kern') and \
> ($msg contains 'IN=' and $msg contains 'OUT=') \ then {
> -/var/log/firewall
the leading - does nothing in rsyslog (in traditional syslog it meant that the
write did not need to be synchronous, but in rsyslog everything is async due to
the queues)
> stop
> }
> if ($programname == 'acpid' or $syslogtag == '[acpid]:') and \
> ($syslogseverity <= 5 /* notice */) \ then {
> -/var/log/acpid
> stop
> }
> if ($programname == 'NetworkManager') or \
> ($programname startswith 'nm-') \ then {
> -/var/log/NetworkManager
> stop
> }
> mail.* -/var/log/mail
> mail.info -/var/log/mail.info
> mail.warning -/var/log/mail.warn
> mail.err /var/log/mail.err
> news.crit -/var/log/news/news.crit
> news.err -/var/log/news/news.err
> news.notice -/var/log/news/news.notice
> *.=warning;*.=err -/var/log/warn
> *.crit /var/log/warn
> *.*;mail.none;news.none -/var/log/messages
> local0.*;local1.* -/var/log/localmessages
> local2.*;local3.* -/var/log/localmessages
> local4.*;local5.* -/var/log/localmessages
> local6.*;local7.* -/var/log/localmessages
>
> Is there a way to also validate the configs in /etc/rsyslog.d/? Or will the
> -N1 also validate those?
-N1 also validates those
I am not seeing anything obvious, unless the trailing backslashes are confusing
things somehow.
but this doesn't match your e-mail blow (where is the fromhost_ip and the
array)
the messags you show below writing to the console look like they are kernel
messages, so your config is writing them out as specified.
remember, it doesn't stop processing the log at the first match, it keeps going
in case there are other matches as well.
David Lang
> -----Original Message-----
> From: rsyslog-boun...@lists.adiscon.com
> [mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of David Lang
> Sent: Friday, December 02, 2016 11:55 AM
> To: rsyslog-users <rsyslog@lists.adiscon.com>
> Subject: Re: [rsyslog] filters question
>
> are you sure there are no other errors in your config? do rsyslogd -N1
> and check for any errors. Once you have errors in the config all best
> are off
>
> David Lang
>
> On Fri, 2 Dec 2016, Swartz, Patrick wrote:
>
>> Date: Fri, 2 Dec 2016 14:40:05 +0000
>> From: "Swartz, Patrick" <patrick.swa...@tyson.com>
>> Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>> Subject: Re: [rsyslog] filters question
>>
>> Okay... I've made some changes to my configs using the output from the
>> debug. I'm now using "fromhost_ip ==" and statically listing every possible
>> IP in the array, and still some messages are falling through to my
>> Unclassified.
>> Probably better to show than to try and explain...
>>
>> From /etc/rsyslogd.conf:
>> ### for debug
>> *.* /var/splunk-syslog/msgdebug.log;RSYSLOG_DebugFormat
>>
>> From debug file:
>> Debug line with all properties:
>> FROMHOST: 'sdcubpe08.mycompany.com', fromhost-ip: '100.20.20.218',
>> HOSTNAME: 'sdcubpe08.mycompany.com', PRI: 167, syslogtag 'Vpxa:',
>> programname: 'Vpxa', APP-NAME: 'Vpxa', PROCID: '-', MSGID: '-',
>> TIMESTAMP: 'Dec 2 14:20:09', STRUCTURED-DATA: '-',
>> msg: ' verbose vpxa[2ACE2B70] [Originator@6876 sub=halservices
>> opID=WFU-32dbe2e3] [VpxaHalServices] HostChanged Event Fired, properties
>> changed [runtime.healthSystemRuntime]'
>> escaped msg: ' verbose vpxa[2ACE2B70] [Originator@6876 sub=halservices
>> opID=WFU-32dbe2e3] [VpxaHalServices] HostChanged Event Fired, properties
>> changed [runtime.healthSystemRuntime]'
>> inputname: imtcp rawmsg: '<167>2016-12-02T14:20:09.131Z
>> sdcubpe08.mycompany.com Vpxa: verbose vpxa[2ACE2B70] [Originator@6876
>> sub=halservices opID=WFU-32dbe2e3] [VpxaHalServices] HostChanged Event
>> Fired, properties changed [runtime.healthSystemRuntime]'
>> $!:
>> $.:
>> $/:
>>
>> From my /etc/rsyslog.d/ESXi.conf
>> template(name="ESXi_app" type="string"
>> string="/var/splunk-syslog/ESXi/%FROMHOST%/%FROMHOST%-%$NOW%.log")
>> if $fromhost-ip == ["100.31.20.101",
>> "100.31.20.102",
>> "100.20.20.218"] ######### I've shortened the list
>> here for list clarity #####
>> then {
>> action(type="omfile" dirCreateMode="0755"
>> FileCreateMode="0644" dynaFile="ESXi_app") stop }
>>
>> Is there a "priority" in how rsyslog reads/merges/loads the different
>> configs between the main config (/etc/rsyslog.conf) and the others like
>> /etc/rsyslog.d/ESXi.conf?
>>
>> One other oddity kinda/sorta related. Messages like these keep writing to
>> the terminal:
>> Message from sysl...@sdcurpe02.mycompany.com at Dec 2 09:34:53 ...
>> localcli: libsmartsata: Not an ATA SMART
>> device:naa.600507680c82811eb80000000000006a
>>
>> Message from sysl...@sdcurpe02.mycompany.com at Dec 2 09:34:54 ...
>> localcli: libsmartsata: Not an ATA SMART
>> device:naa.600507680c82811eb80000000000006b
>>
>>
>> Patrick Swartz
>>
>>
>> -----Original Message-----
>> From: rsyslog-boun...@lists.adiscon.com
>> [mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of David Lang
>> Sent: Thursday, December 01, 2016 3:20 PM
>> To: rsyslog-users <rsyslog@lists.adiscon.com>
>> Subject: Re: [rsyslog] filters question
>>
>> On Thu, 1 Dec 2016, Swartz, Patrick wrote:
>>
>>> Hello,
>>> Confession... I'm still learning rsyslog after many years of working with
>>> syslog-ng. I'm using rsyslog-8.4.0-8.3 on a SLES12.1 system and am trying
>>> to capture my ESXi host logs.
>>>
>>> Here is my current filter for those:
>>>
>>> cat /etc/rsyslog.d/ESXi.conf
>>>
>>> template(name="ESXi_app" type="string"
>>> string="/var/splunk-syslog/ESXi/%FROMHOST%/%FROMHOST%-%$NOW%.log")
>>> if $hostname startswith ["cdcubde",
>>> "sdcubde",
>>> "sdcubpe",
>>> "cdcubpe",
>>> "cdcubdmz",
>>> "cdcurpe",
>>> "sdcurpe"]
>>> then {
>>> action(type="omfile" dirCreateMode="0755" FileCreateMode="0644"
>>> dynaFile="ESXi_app")
>>> }
>>> else {
>>> if $programname contains ["Hostd",
>>> "Vpxa",
>>> "xmlns",
>>> "soapenv",
>>> "cdcubpe02"]
>>> then {
>>> action(type="omfile" dirCreateMode="0755" FileCreateMode="0644"
>>> dynaFile="ESXi_app")
>>> }
>>> stop }
>>>
>>> I added the extra "else/if" because even though the 'startwith' was mostly
>>> working, it wasn't working 100%. And, now even with the extra else/if some
>>> messages are still falling through to my "Unknownl" and I don't understand
>>> why.
>>>
>>> Example message that is falling through -
>>>
>>> cat
>>> Unknown/cdcubdmz01.mycompany.com/cdcubdmz01.mycompany.com-2016-12-01.
>>> l
>>> og
>>> 2016-12-01T10:01:22.690936-05:00 cdcubdmz01.mycompay.com soapenv:
>>> Body><HostImageConfigGetAcceptanceResponse
>>> xmlns='urn:vim25'><returnval>partner</returnval></HostImageConfigGet
>>> A
>>> c
>>> ceptanceResponse></soapenv:Body></soapenv:Envelope>
>>>
>>> I'm using different configs in /etc/rsyslog.d/ for the different filters
>>> (ESXi, FireEye, PaloAlto, etc), then my Unknown filter is in the
>>> /etc/rsyslog.conf file. Is that approach wrong?
>>>
>>> From my rsyslog.conf:
>>> template(name="Unknown" type="string"
>>> string="/var/splunk-syslog/Unknown/%HOSTNAME%/%FROMHOST%-%$NOW%.log"
>>> )
>>> *.* action(type="omfile" dirCreateMode="0755" FileCreateMode="0644"
>>> dynaFile="Unknown")
>>>
>>> Any help would be greatly appreciated.
>>
>> Whenever a filter isn't working as expected, the first thing to do is to
>> look at what the data actually is that you are filtering against 99% of the
>> time the problem is that the variable doesn't contain what you expect it to.
>>
>> in your 'unknown' section, log the data with the template
>> RSYSLOG_DebugFormat to a file and look at what it's writing.
>>
>> Or, since you are just looking at hostname, you could make a custom template
>> that just lists that, say '%hostname% -- %rawmsg%\n'
>>
>> odds are really good that you will see the problems at that point. It may
>> raise the question of 'why did it get parsed that way', but that's where
>> logging rawmsg is so useful.
>>
>> RSYSLOG_DebugFormat shows all the properties.
>>
>> David Lang
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This
>> is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our
>> control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>>
>> ---------------------------------------------------------------------
>> - This email and any files transmitted with it are confidential and
>> intended solely for the use of the addressee. If you are not the intended
>> addressee, then you have received this email in error and any use,
>> dissemination, forwarding, printing, or copying of this email is strictly
>> prohibited. Please notify us immediately of your unintended receipt by reply
>> and then delete this email and your reply. Tyson Foods, Inc. and its
>> subsidiaries and affiliates will not be held liable to any person resulting
>> from the unintended or unauthorized use of any information contained in this
>> email or as a result of any additions or deletions of information originally
>> contained in this email.
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
>> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites
>> beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
>> THAT.
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This
> is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our
> control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
> ----------------------------------------------------------------------
> This email and any files transmitted with it are confidential and intended
> solely for the use of the addressee. If you are not the intended addressee,
> then you have received this email in error and any use, dissemination,
> forwarding, printing, or copying of this email is strictly prohibited. Please
> notify us immediately of your unintended receipt by reply and then delete
> this email and your reply. Tyson Foods, Inc. and its subsidiaries and
> affiliates will not be held liable to any person resulting from the
> unintended or unauthorized use of any information contained in this email or
> as a result of any additions or deletions of information originally contained
> in this email.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites
> beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is
a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our
control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
----------------------------------------------------------------------
This email and any files transmitted with it are confidential and intended
solely for the use of the addressee. If you are not the intended addressee,
then you have received this email in error and any use, dissemination,
forwarding, printing, or copying of this email is strictly prohibited. Please
notify us immediately of your unintended receipt by reply and then delete this
email and your reply. Tyson Foods, Inc. and its subsidiaries and affiliates
will not be held liable to any person resulting from the unintended or
unauthorized use of any information contained in this email or as a result of
any additions or deletions of information originally contained in this email.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
