On Thu, 8 Oct 2009, Martin Mielke wrote: > Hi all, > > odd behavior here... > > Today one of our core systems stopped accepting ssh connections. > -- > $ telnet sshserver 22 > Trying 1.2.3.4... > Connected to sshserver. > Escape character is '^]'. > Connection closed. > -- > > Using a pre-established session rsyslog was shutdown: > -- > # /etc/init.d/rsyslog stop > Shutting down system logger: [ OK > ][/quote] > -- > > ..and the system started accepting connections again: > -- > $ ssh sshserver > Last login: Thu Oct 8 01:12:59 2009 from foo.bar.tld > -- > > We are just one step before dropping rsyslog and use syslog-ng again > (which was dismissed some time ago to favor rsyslog) but I would like to > know what's the reason for this and, most important, what's the solution > :-)
ssh logs connections, if the syslog process cannot process the message, ssh is designed to stop and wait until it does (it deems the log so important that it refuses to do anything until the log is written) conole logins do the same thing. so something causes rsyslog to stop accepting messages. the same thing can happen to syslog-ng or to plain sysklog, just under different conditions. so the question is why rsyslog stopped. what is rsyslog configured to do with log messages? is there any chance that it was unable to do something with a message and so would have had to keep it in it's queue until the queue filled up? (logging via TCP to a remote server that stops responding will do this, so will writing to a full filesystem) rsyslog can be configured to accept and discard log entries when the queue is full, doing this can avoid this sort of situation. David Lang > Some additional info: > -- > # ulimit -a > core file size (blocks, -c) 0 > data seg size (kbytes, -d) unlimited > scheduling priority (-e) 0 > file size (blocks, -f) unlimited > pending signals (-i) 32767 > max locked memory (kbytes, -l) 32 > max memory size (kbytes, -m) unlimited > open files (-n) 1024 > pipe size (512 bytes, -p) 8 > POSIX message queues (bytes, -q) 819200 > real-time priority (-r) 0 > stack size (kbytes, -s) 10240 > cpu time (seconds, -t) unlimited > max user processes (-u) 32767 > virtual memory (kbytes, -v) unlimited > file locks (-x) unlimited > -- > > > Thanks! > > Martin > > > This email and any attachments are confidential, and may be legally > privileged and protected by copyright. If you are not the intended recipient > dissemination or copying of this email is prohibited. If you have received > this in error, please notify the sender by replying by email and then delete > the email completely from your system. > > Any views or opinions are solely those of the sender. This communication is > not intended to form a binding contract unless expressly indicated to the > contrary and properly authorised. Any actions taken on the basis of this > email are at the recipient's own risk. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
