Well, it is always possible that you have some other bottleneck in your systems.
I assumed that you were reporting that the number of connections made a huge
difference, even if the volume of logs remained pretty much the same.
before focusing on the queue configuration (which seldom causes the problem),
the first thing I would be trying to do is to see where rsyslog is spending it's
time.
run top and enable thread views (the 'H' key) and see if one of the threads is
using 100% CPU or something close to that. If so, we will want to figure out
what that thread is doing.
If it's the imtcp thread, then you could be running into an input limit. But if
it's one of the output threads, then the problem is that with your
configuration, rsyslog is not able to process the messages as fast as they are
arriving. That will then cause the queue to fill up and devices sending via TCP
to block. Increasing the queue size will probably not help (the large queue may
take a few more seconds to fill, but will still fill)
Instead we would need to look at optimizing the processing of the messages.
David Lang
On Fri, 14 Jun 2013, Timothy Ehlers wrote:
I think the queue is not properly configured could this backup to not
allowing connections?
rsyslogd-pstats: main Q: size=39999989 enqueued=98847738 full=1805696
discarded.full=0 discarded.nf=0 maxqsize=40000000
On Fri, Jun 14, 2013 at 1:51 PM, David Lang <[email protected]> wrote:
take a look at the imptcp module. My understanding is that it handles
large numbers of connections better than the plain imtcp module
I have done very high traffic volume tests with rsyslog, but have not been
in a position to test thousands of connections to one box. Rainer will need
to comment on this.
David Lang
On Fri, 14 Jun 2013, Timothy Ehlers wrote:
Is there a recommended connection count per port.
We are sending to rsyslog using syslog-ng and this was working for a
while.
Suddenly syslog-ng is sending data which rsyslog is not acknowledging
(tcpdump).
We have a very high server count and were at ~3000 connections on port
20514, currently rsyslog seems to drag after ~500 connections. But it is
slowly building them back up.
Are we hitting some kind of wall?
Version info:
rsyslog-7.2.5
Ulimit info:
ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 385913
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 65536
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 1024
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
______________________________**_________________
rsyslog mailing list
http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>
http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/>
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
