Hello, I'm newish to the world of rsyslog. I've used it for regular syslogging stuff with dynamically generated filenames and the like, but I've never gone in depth with its configuration. Now, however, I'm working on a project that will involve very high message rates and am trying to figure out how to tune the system for the kind of throughput (>200k/sec) documented at http://mperedim.wordpress.com/2010/01/21/rsyslog-evaluation/
I have a pair of hardware test servers outfitted with dual quad-core Xeon processors and 8GB of RAM. The network connection is only 100Mbit, but that doesn't seem to be my bottleneck yet. I'm using loggen for my tests. The servers are running rsyslog 5.5.6. serverB is the one listening for connections. It's using the configuration pasted below. serverA is the one running the test: # loggen --verbose -r 20000 -I 10 -s 512 -S 207.150.202.100 10514 average rate = 5663.95 msg/sec, count=56644, time=10.007, (last) msg size=512, bandwidth=2831.98 kB/sec Increasing the rate (-r) doesn't change the average rate. Switching to UDP or performing the test on localhost yield very similar results. What's really strange is that I left the servers alone for about an hour to work on another project and when I came back the rates were roughly double - 10-13k/sec. As I tested, however, they gradually fell back to the 5-6k levels you see here. Restarting the rsyslog process makes no difference. So I've tuned something incorrectly but I have no idea what. iperf shows 100Mbit between the servers. I can double or halve the message size without affecting the rate, so actual message rate is the problem -- not bandwidth. Any help would be wonderful. -SH # rsyslog.conf: $FileOwner syslog $FileGroup adm $FileCreateMode 0640 $DirCreateMode 0755 $Umask 0022 # different rulesets even though I haven't seen any performance effects $RuleSet remote10514 $RulesetCreateMainQueue on # create ruleset-specific queue $MainMsgQueueSize 100000 $MainMsgQueueDequeueBatchSize 1024 $RepeatedMsgReduction off *.* /dev/null & ~ $ModLoad imtcp $InputTCPServerBindRuleset remote10514 $InputTCPServerRun 10514 $RuleSet RSYSLOG_DefaultRuleset $RepeatedMsgReduction off *.* /var/log/test.log _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
