I have also done a test lab, and I can't confirm these bad numbers. I used Ubuntu 14.04LTS, because that I had readily at hand. I also used a simple testing tool and not openldap, which would be pretty cumbersome for me to setup. I used a somewhat dated desktop machines (2..4 years old) where Ubuntu ran inside a VM.
I did two tests, one with writing to a file, one with no actions (just the discard). I sent one million messages and timed it (inprecisely) with the time command. file write ------------ took consistently between 14 and 12 seconds to get through. Main queue grew up to a max of ~100 msgs. empty conf (just discard) ----------------------------------- numbers varied greatly, some extremes are 18 and 38 seconds. Main queue grew up to max of ~15msgs. In the empty conf case, there is no real processing done, and so we have a lot of locking contention on the main queue, which is the reason for the slower execution time and the inconsistency in it. Bottom line: while imuxsock is slower than imtcp (I would expect around half a million messages on that setup, maybe more), it performs considerably well. Question now: why? I see two potential trouble spots: a) OS b) openldap So we need some additional testing from the OP. As this is the simplest, I would replace openldap by rsyslog's syslog caller (inside ./tests). On an otherwise idle system, do $ ./syslog_caller -m10000 to see what happens. If it is quick, I suggest doing a million or more of messages to get a steady flow. If it is quick, there is a problem with how openldap writes the logs and the interaction with it. If it is not quicker, you need to repeat the issue on Ubuntu 14.04 LTS. If it then is quick, try openldap. If it, too, is quick, we know there is a platform problem. If the test on Ubuntu 14.04LTS is also slow, there is some environment dependency I do not know about. That would require case-specific consulting. Please let me know your results. HTH Rainer 2015-01-12 10:12 GMT+01:00 Rainer Gerhards <[email protected]>: > I created an issue tracker for this: > https://github.com/rsyslog/rsyslog/issues/204 > > 2014-12-23 21:33 GMT+01:00 Marc Fournier <[email protected]>: > >> Excerpts from David Lang's message of 2014-12-21 07:24:36 +0100: >> > On Sat, 20 Dec 2014, Marc Fournier wrote: >> > >> > > Excerpts from David Lang's message of 2014-12-19 18:25:27 +0100: >> > >> >> > >> On Fri, 19 Dec 2014, Marc Fournier wrote: >> > >> >> > >>> - no rsyslog running: 15k to 17k ldap operations/second >> > >>> - rsyslog with above config, except for MainMsgQueueWorkerThreads >> back >> > >>> to 1: down to 7.5k ops/s >> > >> >> > >> wihtout rsyslog running, there shouldn't even be a /dev/log to write >> to, so any >> > >> logging should be a noop. >> > >> >> > >> are you running systemd by any chance? >> > > >> > > No. All this is on an up to date rhel6, which predates systemd. >> > > >> > > I basically wanted to compare the throughput of openldap with vs >> without >> > > logging. >> > > >> > > My point here is that openldap has no way of knowing that nothing is >> > > reading /dev/log. The same code paths leading to syslog() are used, >> etc. >> > > It apparently just suffers from the performance issues downstream. >> > >> > well, if rsyslog isn't running and /dev/log doesn't exist, openldap may >> be >> > short-circuting the logging output as it detects that there isn't >> anyplace to >> > write to >> >> Browsing through openldap's code, the logging part seems pretty basic. >> One call to openlog() at initialisation time, then calls to syslog() via >> wrapper functions. I'm not very familiar with this codebase, but it >> really doesn't seem to do anything fancy like you suggest. >> >> I tried running openldap with the "loglevel none" directive, and I reach >> 30k ops/sec (as opposed to 15k with logging enabled but no /dev/log). So >> if openldap does some sort of short-circuit in this case, there >> definitely is room for improvement... >> >> >> > >> let's simplify the config more, try making it just >> > >> >> > >> module(load="impstats" log.syslog="off" >> log.file="/var/log/impstats.log" resetCounters="off" interval="5") >> > >> module(load="imuxsock" SysSock.RateLimit.Interval="0") >> > >> $MaxMessageSize 2k >> > >> $MainMsgQueueDequeueBatchSize 4096 >> > >> $RepeatedMsgReduction off >> > >> >> > >> *.* ~ >> > >> >> > >> The next step I would try is to change the parser settings to avoid >> wasting >> > >> effort trying to parse the message, but I would be a bit surprised >> if that was >> > >> the problem here. >> > > >> > > Thanks ! I'll give this a try on monday. >> >> I tried this. It performed slightly less well (around -10%) than the >> version with "$MainMsgQueueType LinkedList" I reported the other day. >> >> Thanks ! >> >> Cheers, >> Marc >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
