This sounds very strange, even the early v4 version could work at higher rates. Do you use 512 byte messages, only? Could you start with a very basic rsyslog.conf to get a baseline? Just loading imtcp, starting the listener, and a single
*.* /path/to/some/file Config. Rainer > -----Original Message----- > From: [email protected] [mailto:rsyslog- > [email protected]] On Behalf Of S H > Sent: Saturday, August 07, 2010 1:39 AM > To: [email protected] > Subject: [rsyslog] Tuning for performance > > Hello, > > I'm newish to the world of rsyslog. I've used it for regular > syslogging stuff with dynamically generated filenames and the like, > but I've never gone in depth with its configuration. Now, however, I'm > working on a project that will involve very high message rates and am > trying to figure out how to tune the system for the kind of throughput > (>200k/sec) documented at > http://mperedim.wordpress.com/2010/01/21/rsyslog-evaluation/ > > I have a pair of hardware test servers outfitted with dual quad-core > Xeon processors and 8GB of RAM. The network connection is only > 100Mbit, but that doesn't seem to be my bottleneck yet. I'm using > loggen for my tests. The servers are running rsyslog 5.5.6. > > serverB is the one listening for connections. It's using the > configuration pasted below. > > serverA is the one running the test: > # loggen --verbose -r 20000 -I 10 -s 512 -S 207.150.202.100 10514 > average rate = 5663.95 msg/sec, count=56644, time=10.007, (last) msg > size=512, bandwidth=2831.98 kB/sec > > Increasing the rate (-r) doesn't change the average rate. Switching to > UDP or performing the test on localhost yield very similar results. > What's really strange is that I left the servers alone for about an > hour to work on another project and when I came back the rates were > roughly double - 10-13k/sec. As I tested, however, they gradually fell > back to the 5-6k levels you see here. Restarting the rsyslog process > makes no difference. So I've tuned something incorrectly but I have no > idea what. > > iperf shows 100Mbit between the servers. I can double or halve the > message size without affecting the rate, so actual message rate is the > problem -- not bandwidth. > > Any help would be wonderful. > > -SH > > # rsyslog.conf: > > $FileOwner syslog > $FileGroup adm > $FileCreateMode 0640 > $DirCreateMode 0755 > $Umask 0022 > > # different rulesets even though I haven't seen any performance effects > $RuleSet remote10514 > $RulesetCreateMainQueue on # create ruleset-specific queue > $MainMsgQueueSize 100000 > $MainMsgQueueDequeueBatchSize 1024 > $RepeatedMsgReduction off > > *.* /dev/null > & ~ > > $ModLoad imtcp > $InputTCPServerBindRuleset remote10514 > $InputTCPServerRun 10514 > > > $RuleSet RSYSLOG_DefaultRuleset > $RepeatedMsgReduction off > > *.* /var/log/test.log > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
