I am trying to translate log flow from syslog-ng to rsyslog.
I have a few syslog sources that collect logs from all over the
infrastructure and deliver it to central syslog servers that in turn
deliver logs to different consumers.
At the moment, I have two consumers - one reads flat files and other
listens on TCP ports. For both, I need to tell them what type of event am I
delivering to them - Linux auth, firewall event, or web log.
Something like this:
LogFireHose====> SyslogNG/RSyslog =========>
(Parse and redirect events by type)||==> If (Cisco
ASA) write to "FS (Cisco/ASA) & TCP DstHost: 5000"
||==> If (Apache Access) write to "FS (Apache/Access) & TCP DstHost: 5001"
||==> If (DNS logs) write to "FS (Bind/DNS) & TCP DstHost: 5002"
In Syslog-NG, every incoming message (in CEF format) is subject to a parser
that splits the log message into eight fields. Fields 2 and 3 that are
vendor and product type are used to generate a template like
"/var/log/$vendor/$product/logfile".
To deliver events, by type, to a specific network destination requires
filters and I have 30+ different vendor/product combinations. So I end up
with 30+ log() statements, each with it's filter logic.
----------xxxxxxxxxxxxxxx----------------
filter f1 (if $product contains "ASA")
filter f2 (if $product contains "ACCESS")
filter f3 (if $product contains "DNS")
...
..
filter 35 (if field3 contains "blah")
log (src=tcp;filter f1; dst=/var/log/$vendor/$product/logfile;
dst=remotehost:5000)
log (src=tcp;filter f2; dst=/var/log/$vendor/$product/logfile;
dst=remotehost:5001)
...
....
log (src=tcp;filter fx, dst=/var/log/$vendor/$product/logfile;
dst=remotehost:5030)
----------xxxxxxxxxxxxxxx----------------
In RSyslog, I have so far written the logic to write to filesystem like
this:
--------------xxxxxxxxxxxxx---------------
template(name="cefdynfile" type="string"
string="/var/log/joe/%msg:F,124:2%/%msg:F,124:3%/logfile")
ruleset(name="tcpcef") {
if $syslogtag=="CEF:" then { action (type="omfile" FileOwner="joe"
FileGroup="joe" DirOwner="joe" DirGroup="joe" DirCreateMode="0755"
FileCreateMode="0644" DynaFile="cefdynfile") stop }
}
module(load="imtcp") # needs to be done just once
input(type="imtcp" port="514" ruleset="tcpcef")
--------------xxxxxxxxxxxxx---------------
Now, I am thinking how do I add rules for delivering events to tcp
destinations.
I could expand the "tcpcef" ruleset and add more "if condition then
{action()}" statements to it, OR
I can write multiple rulesets, one for each filter like
"rule f1 { if $msg contains "blah" then action()}"
"rule f2 { if $msg contains "foo" then action()}"
and then call these rules from "tcpcef" ruleset:
ruleset(name="tcpcef") {
call f1
call f2
...
...
call fx }
So two questions (1) Does this seem like a good way to parse/route
messages?
(2) Which way is better for multi-threading? I read that each ruleset gets
it own queue and thread so I am thinking defining multiple rulesets and
then calling them for a master ruleset might offer better performance.
Thanks for patiently reading through this email even if you do not respond
:-)
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
