On Fri, 2 Dec 2016, Swartz, Patrick wrote:
root@whqlrsyslog01 # rsyslogd -N1
rsyslogd: version 8.4.0, config validation run (level 1), master config
/etc/rsyslog.conf
rsyslogd: End of config validation run. Bye.
Here is my full rsyslogd.conf (minus comments). My additions/changes are
marked to the side (the comments are not in the file), everything else is stock
from SUSE.
root@whqlrsyslog01 # sed -e '/\s*#.*$/d' -e '/^\s*$/d' /etc/rsyslog.conf
$umask 0000 ######### Added by pswartz
*.* /var/splunk-syslog/msgdebug.log;RSYSLOG_DebugFormat
######### Added by pswartz
$ModLoad immark.so
$MarkMessagePeriod 3600
$ModLoad imuxsock.so
$RepeatedMsgReduction on
we actually recommend not doing message reduction, it's a lot easier for
monitoring to deal with the same log message 500 times than a log message that
says 'last message repeated 499 times'
$ModLoad imklog.so
$klogConsoleLogLevel 1
$IncludeConfig /etc/rsyslog.d/*.conf ######### Added by pswartz
$IncludeConfig /etc/rsyslog.d/*.template ######### Added by
pswartz
$umask 0000
template(name="Unclassified" type="string"
string="/var/splunk-syslog/Unclassified/%HOSTNAME%/%FROMHOST%-%$NOW%.log") ##### Added by
pswartz
*.* action(type="omfile" dirCreateMode="0755" FileCreateMode="0644"
dynaFile="Unclassified") ######### Added by pswartz
if ( \
are these trailing slashes in the file? they should not be needed
/* kernel up to warning except of firewall */ \
I hope this is one of the comments you say are not actually in the file.
($syslogfacility-text == 'kern') and \
($syslogseverity <= 4 /* warning */ ) and not \
($msg contains 'IN=' and $msg contains 'OUT=') \
) or ( \
/* up to errors except of facility authpriv */ \
($syslogseverity <= 3 /* errors */ ) and not \
($syslogfacility-text == 'authpriv') \
) \
then {
/dev/tty10
|/dev/xconsole
}
*.emerg :omusrmsg:*
if ($syslogfacility-text == 'kern') and \
($msg contains 'IN=' and $msg contains 'OUT=') \
then {
-/var/log/firewall
the leading - does nothing in rsyslog (in traditional syslog it meant that the
write did not need to be synchronous, but in rsyslog everything is async due to
the queues)
stop
}
if ($programname == 'acpid' or $syslogtag == '[acpid]:') and \
($syslogseverity <= 5 /* notice */) \
then {
-/var/log/acpid
stop
}
if ($programname == 'NetworkManager') or \
($programname startswith 'nm-') \
then {
-/var/log/NetworkManager
stop
}
mail.* -/var/log/mail
mail.info -/var/log/mail.info
mail.warning -/var/log/mail.warn
mail.err /var/log/mail.err
news.crit -/var/log/news/news.crit
news.err -/var/log/news/news.err
news.notice -/var/log/news/news.notice
*.=warning;*.=err -/var/log/warn
*.crit /var/log/warn
*.*;mail.none;news.none -/var/log/messages
local0.*;local1.* -/var/log/localmessages
local2.*;local3.* -/var/log/localmessages
local4.*;local5.* -/var/log/localmessages
local6.*;local7.* -/var/log/localmessages
Is there a way to also validate the configs in /etc/rsyslog.d/? Or will the -N1
also validate those?
-N1 also validates those
I am not seeing anything obvious, unless the trailing backslashes are confusing
things somehow.
but this doesn't match your e-mail blow (where is the fromhost_ip and the
array)
the messags you show below writing to the console look like they are kernel
messages, so your config is writing them out as specified.
remember, it doesn't stop processing the log at the first match, it keeps going
in case there are other matches as well.
David Lang
-----Original Message-----
From: rsyslog-boun...@lists.adiscon.com
[mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of David Lang
Sent: Friday, December 02, 2016 11:55 AM
To: rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] filters question
are you sure there are no other errors in your config? do rsyslogd -N1 and
check for any errors. Once you have errors in the config all best are off
David Lang
On Fri, 2 Dec 2016, Swartz, Patrick wrote:
Date: Fri, 2 Dec 2016 14:40:05 +0000
From: "Swartz, Patrick" <patrick.swa...@tyson.com>
Reply-To: rsyslog-users <rsyslog@lists.adiscon.com>
To: rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] filters question
Okay... I've made some changes to my configs using the output from the debug. I'm now
using "fromhost_ip ==" and statically listing every possible IP in the array,
and still some messages are falling through to my Unclassified.
Probably better to show than to try and explain...
From /etc/rsyslogd.conf:
### for debug
*.* /var/splunk-syslog/msgdebug.log;RSYSLOG_DebugFormat
From debug file:
Debug line with all properties:
FROMHOST: 'sdcubpe08.mycompany.com', fromhost-ip: '100.20.20.218',
HOSTNAME: 'sdcubpe08.mycompany.com', PRI: 167, syslogtag 'Vpxa:',
programname: 'Vpxa', APP-NAME: 'Vpxa', PROCID: '-', MSGID: '-',
TIMESTAMP: 'Dec 2 14:20:09', STRUCTURED-DATA: '-',
msg: ' verbose vpxa[2ACE2B70] [Originator@6876 sub=halservices
opID=WFU-32dbe2e3] [VpxaHalServices] HostChanged Event Fired, properties
changed [runtime.healthSystemRuntime]'
escaped msg: ' verbose vpxa[2ACE2B70] [Originator@6876 sub=halservices
opID=WFU-32dbe2e3] [VpxaHalServices] HostChanged Event Fired, properties
changed [runtime.healthSystemRuntime]'
inputname: imtcp rawmsg: '<167>2016-12-02T14:20:09.131Z sdcubpe08.mycompany.com
Vpxa: verbose vpxa[2ACE2B70] [Originator@6876 sub=halservices opID=WFU-32dbe2e3]
[VpxaHalServices] HostChanged Event Fired, properties changed
[runtime.healthSystemRuntime]'
$!:
$.:
$/:
From my /etc/rsyslog.d/ESXi.conf
template(name="ESXi_app" type="string"
string="/var/splunk-syslog/ESXi/%FROMHOST%/%FROMHOST%-%$NOW%.log")
if $fromhost-ip == ["100.31.20.101",
"100.31.20.102",
"100.20.20.218"] ######### I've shortened the list
here for list clarity #####
then {
action(type="omfile" dirCreateMode="0755"
FileCreateMode="0644" dynaFile="ESXi_app") stop }
Is there a "priority" in how rsyslog reads/merges/loads the different configs
between the main config (/etc/rsyslog.conf) and the others like /etc/rsyslog.d/ESXi.conf?
One other oddity kinda/sorta related. Messages like these keep writing to the
terminal:
Message from sysl...@sdcurpe02.mycompany.com at Dec 2 09:34:53 ...
localcli: libsmartsata: Not an ATA SMART
device:naa.600507680c82811eb80000000000006a
Message from sysl...@sdcurpe02.mycompany.com at Dec 2 09:34:54 ...
localcli: libsmartsata: Not an ATA SMART
device:naa.600507680c82811eb80000000000006b
Patrick Swartz
-----Original Message-----
From: rsyslog-boun...@lists.adiscon.com
[mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of David Lang
Sent: Thursday, December 01, 2016 3:20 PM
To: rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] filters question
On Thu, 1 Dec 2016, Swartz, Patrick wrote:
Hello,
Confession... I'm still learning rsyslog after many years of working with
syslog-ng. I'm using rsyslog-8.4.0-8.3 on a SLES12.1 system and am trying to
capture my ESXi host logs.
Here is my current filter for those:
cat /etc/rsyslog.d/ESXi.conf
template(name="ESXi_app" type="string"
string="/var/splunk-syslog/ESXi/%FROMHOST%/%FROMHOST%-%$NOW%.log")
if $hostname startswith ["cdcubde",
"sdcubde",
"sdcubpe",
"cdcubpe",
"cdcubdmz",
"cdcurpe",
"sdcurpe"]
then {
action(type="omfile" dirCreateMode="0755" FileCreateMode="0644"
dynaFile="ESXi_app")
}
else {
if $programname contains ["Hostd",
"Vpxa",
"xmlns",
"soapenv",
"cdcubpe02"]
then {
action(type="omfile" dirCreateMode="0755" FileCreateMode="0644"
dynaFile="ESXi_app")
}
stop }
I added the extra "else/if" because even though the 'startwith' was mostly working, it
wasn't working 100%. And, now even with the extra else/if some messages are still falling through
to my "Unknownl" and I don't understand why.
Example message that is falling through -
cat
Unknown/cdcubdmz01.mycompany.com/cdcubdmz01.mycompany.com-2016-12-01.
l
og
2016-12-01T10:01:22.690936-05:00 cdcubdmz01.mycompay.com soapenv:
Body><HostImageConfigGetAcceptanceResponse
xmlns='urn:vim25'><returnval>partner</returnval></HostImageConfigGetA
c
ceptanceResponse></soapenv:Body></soapenv:Envelope>
I'm using different configs in /etc/rsyslog.d/ for the different filters
(ESXi, FireEye, PaloAlto, etc), then my Unknown filter is in the
/etc/rsyslog.conf file. Is that approach wrong?
From my rsyslog.conf:
template(name="Unknown" type="string"
string="/var/splunk-syslog/Unknown/%HOSTNAME%/%FROMHOST%-%$NOW%.log")
*.* action(type="omfile" dirCreateMode="0755" FileCreateMode="0644"
dynaFile="Unknown")
Any help would be greatly appreciated.
Whenever a filter isn't working as expected, the first thing to do is to look
at what the data actually is that you are filtering against 99% of the time the
problem is that the variable doesn't contain what you expect it to.
in your 'unknown' section, log the data with the template RSYSLOG_DebugFormat
to a file and look at what it's writing.
Or, since you are just looking at hostname, you could make a custom template
that just lists that, say '%hostname% -- %rawmsg%\n'
odds are really good that you will see the problems at that point. It may raise
the question of 'why did it get parsed that way', but that's where logging
rawmsg is so useful.
RSYSLOG_DebugFormat shows all the properties.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is
a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our
control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
----------------------------------------------------------------------
This email and any files transmitted with it are confidential and intended
solely for the use of the addressee. If you are not the intended addressee,
then you have received this email in error and any use, dissemination,
forwarding, printing, or copying of this email is strictly prohibited. Please
notify us immediately of your unintended receipt by reply and then delete this
email and your reply. Tyson Foods, Inc. and its subsidiaries and affiliates
will not be held liable to any person resulting from the unintended or
unauthorized use of any information contained in this email or as a result of
any additions or deletions of information originally contained in this email.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites
beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is
a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our
control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
----------------------------------------------------------------------
This email and any files transmitted with it are confidential and intended
solely for the use of the addressee. If you are not the intended addressee,
then you have received this email in error and any use, dissemination,
forwarding, printing, or copying of this email is strictly prohibited. Please
notify us immediately of your unintended receipt by reply and then delete this
email and your reply. Tyson Foods, Inc. and its subsidiaries and affiliates
will not be held liable to any person resulting from the unintended or
unauthorized use of any information contained in this email or as a result of
any additions or deletions of information originally contained in this email.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
