hi rsyslogers: we've been doing some load testing of syslog messages over UDP/514 to rsyslog. we write all incoming messages to a file in /data/logs/incoming-all.log.
In our load test, we generated about 29 million messages in 300 seconds. On the server side, we are receiving about 25 million messages; and about 4 million messages are lost on the network (not an rsyslog issue). However, of the 25 million messages we know arrive at the server, we are also seeing message lost in /data/logs/incoming-all.log, albeit to a much lesser degree than the network problem. The actual numbers are: 29,561,113 messages generated and sent in 300 seconds 24,802,441 messages arrive at the rsyslog server (counting UDP packets via NETFILTER/mangle-PREROUTING accounting rule) 24,774,587 messages written to /data/logs/incoming-all.log So it would seem that we lost 27854 messages within rsyslog. My question is this: 1. Does rsyslog drop messages when its message queues are overflowing? 2. If answer to #1 is yes, does it keep any accounting of the lost messages and how can I see those numbers? or at least warn that its queues are overflowing? 3. if answer to #1 is yes, is there some configuration setting to make rsyslog guarantee not to drop messages, potentially as trade off with some other problem? Or is it just a matter of increasing queue sizes? 4. If answer to #1 is no, what's the best way to go about troubleshooting why messages are being lost? BTW, under less stressful conditions, all the in/out numbers perfectly match. We only start seeing "lost messages/packets" when we go above ~50,000 messages per second. Would appreciate any insights... -Bond _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
