Re: [rsyslog] filters question

[David Lang]

On Thu, 1 Dec 2016, Swartz, Patrick wrote:

Hello,
Confession... I'm still learning rsyslog after many years of working with 
syslog-ng.  I'm using rsyslog-8.4.0-8.3 on a SLES12.1 system and am trying to 
capture my ESXi host logs.
Here is my current filter for those:

cat /etc/rsyslog.d/ESXi.conf

template(name="ESXi_app" type="string" 
string="/var/splunk-syslog/ESXi/%FROMHOST%/%FROMHOST%-%$NOW%.log")
if $hostname startswith ["cdcubde",
                   "sdcubde",
                   "sdcubpe",
                   "cdcubpe",
                   "cdcubdmz",
                   "cdcurpe",
                   "sdcurpe"]
  then {
        action(type="omfile" dirCreateMode="0755" FileCreateMode="0644" 
dynaFile="ESXi_app")
       }
else {
if $programname contains ["Hostd",
                   "Vpxa",
                   "xmlns",
                   "soapenv",
                   "cdcubpe02"]
  then {
       action(type="omfile" dirCreateMode="0755" FileCreateMode="0644" 
dynaFile="ESXi_app")
       }
stop }

I added the extra "else/if" because even though the 'startwith' was mostly working, it 
wasn't working 100%.  And, now even with the extra else/if some messages are still falling through 
to my "Unknownl" and I don't understand why.

Example message that is falling through -

cat Unknown/cdcubdmz01.mycompany.com/cdcubdmz01.mycompany.com-2016-12-01.log
2016-12-01T10:01:22.690936-05:00 cdcubdmz01.mycompay.com soapenv: Body><HostImageConfigGetAcceptanceResponse 
xmlns='urn:vim25'><returnval>partner</returnval></HostImageConfigGetAcceptanceResponse></soapenv:Body></soapenv:Envelope>

I'm using different configs in /etc/rsyslog.d/  for the different filters 
(ESXi, FireEye, PaloAlto, etc), then my Unknown filter is in the 
/etc/rsyslog.conf file.  Is that approach wrong?

From my rsyslog.conf:
template(name="Unknown" type="string" 
string="/var/splunk-syslog/Unknown/%HOSTNAME%/%FROMHOST%-%$NOW%.log")
*.* action(type="omfile" dirCreateMode="0755" FileCreateMode="0644" 
dynaFile="Unknown")

Any help would be greatly appreciated.

Whenever a filter isn't working as expected, the first thing to do is to look at 
what the data actually is that you are filtering against 99% of the time the 
problem is that the variable doesn't contain what you expect it to.

in your 'unknown' section, log the data with the template RSYSLOG_DebugFormat to 
a file and look at what it's writing.

Or, since you are just looking at hostname, you could make a custom template 
that just lists that, say '%hostname% -- %rawmsg%\n'

odds are really good that you will see the problems at that point. It may raise 
the question of 'why did it get parsed that way', but that's where logging 
rawmsg is so useful.

RSYSLOG_DebugFormat shows all the properties.

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.