I've also had problems generating enough traffic to stress rsyslog, what I ended up doing is using tcpreplay. I send out sample data, capture it with tcpdump, and then use tcpreplay to replay the packets over the network at different speeds.
this only works well for UDP packets (as tcpreplay does not maintain all the state needed to do a real tcp connection.) David Lang On Sat, 7 Aug 2010, S H wrote: > Date: Sat, 7 Aug 2010 11:10:46 -0400 > From: S H <[email protected]> > Reply-To: rsyslog-users <[email protected]> > To: rsyslog-users <[email protected]> > Subject: Re: [rsyslog] Tuning for performance > > I think the problem is with loggen, or at least the version I'm using. > I wrote a simple little utility in Python to exercise the native > syslog() calls and was seeing about 240k/second with a simple > configuration. > > Once I've got more detailed benchmarks, I'll post them to this thread. > Thanks for the help. > > -SH > > On Sat, Aug 7, 2010 at 4:14 AM, Rainer Gerhards > <[email protected]> wrote: >> This sounds very strange, even the early v4 version could work at higher >> rates. Do you use 512 byte messages, only? Could you start with a very basic >> rsyslog.conf to get a baseline? Just loading imtcp, starting the listener, >> and a single >> >> *.* /path/to/some/file >> >> Config. >> >> Rainer >> >>> -----Original Message----- >>> From: [email protected] [mailto:rsyslog- >>> [email protected]] On Behalf Of S H >>> Sent: Saturday, August 07, 2010 1:39 AM >>> To: [email protected] >>> Subject: [rsyslog] Tuning for performance >>> >>> Hello, >>> >>> I'm newish to the world of rsyslog. I've used it for regular >>> syslogging stuff with dynamically generated filenames and the like, >>> but I've never gone in depth with its configuration. Now, however, I'm >>> working on a project that will involve very high message rates and am >>> trying to figure out how to tune the system for the kind of throughput >>> (>200k/sec) documented at >>> http://mperedim.wordpress.com/2010/01/21/rsyslog-evaluation/ >>> >>> I have a pair of hardware test servers outfitted with dual quad-core >>> Xeon processors and 8GB of RAM. The network connection is only >>> 100Mbit, but that doesn't seem to be my bottleneck yet. I'm using >>> loggen for my tests. The servers are running rsyslog 5.5.6. >>> >>> serverB is the one listening for connections. It's using the >>> configuration pasted below. >>> >>> serverA is the one running the test: >>> # loggen --verbose -r 20000 -I 10 -s 512 -S 207.150.202.100 10514 >>> average rate = 5663.95 msg/sec, count=56644, time=10.007, (last) msg >>> size=512, bandwidth=2831.98 kB/sec >>> >>> Increasing the rate (-r) doesn't change the average rate. Switching to >>> UDP or performing the test on localhost yield very similar results. >>> What's really strange is that I left the servers alone for about an >>> hour to work on another project and when I came back the rates were >>> roughly double - 10-13k/sec. As I tested, however, they gradually fell >>> back to the 5-6k levels you see here. Restarting the rsyslog process >>> makes no difference. So I've tuned something incorrectly but I have no >>> idea what. >>> >>> iperf shows 100Mbit between the servers. I can double or halve the >>> message size without affecting the rate, so actual message rate is the >>> problem -- not bandwidth. >>> >>> Any help would be wonderful. >>> >>> -SH >>> >>> # rsyslog.conf: >>> >>> $FileOwner syslog >>> $FileGroup adm >>> $FileCreateMode 0640 >>> $DirCreateMode 0755 >>> $Umask 0022 >>> >>> # different rulesets even though I haven't seen any performance effects >>> $RuleSet remote10514 >>> $RulesetCreateMainQueue on # create ruleset-specific queue >>> $MainMsgQueueSize 100000 >>> $MainMsgQueueDequeueBatchSize 1024 >>> $RepeatedMsgReduction off >>> >>> *.* /dev/null >>> & ~ >>> >>> $ModLoad imtcp >>> $InputTCPServerBindRuleset remote10514 >>> $InputTCPServerRun 10514 >>> >>> >>> $RuleSet RSYSLOG_DefaultRuleset >>> $RepeatedMsgReduction off >>> >>> *.* /var/log/test.log >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
