Hi Everyone,
I have rsyslog 4.4.1 chugging along reasonably well but am looking for ways
to improve performance and optimize the filter ruleset. Unfortunately I have
to create fairly extensive rulesets to filter on hostname, programname,
facility, priority, etc. Some log sources generate a high volume of logs (a
few Mbytes/sec) across multiple machines and others generate a fairly
routine amount of log data - maybe 5 meg per day. Many filters have
duplicate conditions for some values but there is always variance. I have
tried to order the rules so that the highest volume logs match first and
then are discarded. I've included a sample of the rules used for my highest
volume logs (names changed to protect the innocent).
If there are ways to chain or nest rules so that I can take advantage of
matches already made against a log entry to filter it minimally that would
be great. For example, most of the below rules filter on the same facility &
list of hostnames but look for different values in the 'rawmsg'. If I could
filter on the facility & hostname once and then rawmsg to sort to different
destinations I'm guessing it would be lower overhead but I don't really know
how the processing logic works.
Also - if a condition is not met, are other parts of the filter evaluated?
For example, if a message was received on local0, would any conditions
beyond "if $syslogfacility-text == 'local1'" be evaluated? Is it more
efficient to filter on the undecoded value syslogfacility vs.
syslogfacility-text?
I'm looking for suggestions or general techniques for optimizing rule
performance under these circumstances.
$template XMLFormat, "%syslogtag%%msg%\n"
if $syslogfacility-text == 'local1' and ( \
$fromhost startswith 'hosta' or \
$fromhost startswith 'hostb' or \
$fromhost startswith 'hostc' or \
$fromhost startswith 'hostd' \
) and $rawmsg contains 'protocolLogRecord' then
-/log/syslog/collated/server/protocol.log;XMLFormat
& ~ # discard after match
if $syslogfacility-text == 'local1' and ( \
$fromhost startswith 'hosta' or \
$fromhost startswith 'hostb' or \
$fromhost startswith 'hostc' or \
$fromhost startswith 'hostd' \
) and $rawmsg contains 'messageLogRecord' then
-/log/syslog/collated/server/message.log;XMLFormat
& ~ # discard after match
if $syslogfacility-text == 'local1' and ( \
$fromhost startswith 'hosta' or \
$fromhost startswith 'hostb' or \
$fromhost startswith 'hostc' or \
$fromhost startswith 'hostd' \
) and $rawmsg contains 'clientLogRecord' then
-/log/syslog/collated/server/client.log;XMLFormat
& ~ # discard after match
if $syslogfacility-text == 'local2' and ( \
$fromhost startswith 'hosta' or \
$fromhost startswith 'hostb' or \
$fromhost startswith 'hostc' or \
$fromhost startswith 'hostd' \
) then -/log/syslog/collated/server/usage.log;XMLFormat
& ~ # discard after match
if $syslogfacility-text == 'local1' and ( \
$fromhost startswith 'hosta' or \
$fromhost startswith 'hostb' or \
$fromhost startswith 'hostc' or \
$fromhost startswith 'hostd' \
) and $rawmsg contains 'WAP Page Service ID' then
-/log/syslog/collated/server/customer-service;XMLFormat
& ~ # discard after match
if $syslogfacility-text == 'local1' and ( \
$fromhost startswith 'hosta' or \
$fromhost startswith 'hostb' or \
$fromhost startswith 'hostc' or \
$fromhost startswith 'hostd' \
) and $rawmsg contains 'locationlogrecord' then
-/log/syslog/collated/server/lbs.log;XMLFormat
& ~ # discard after match
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
