A "problem" I am aware of is that a died peer (or connection dropped an interim firewall) is not detected as broken, because no messages are exchanged any longer. An often-used solution is KEEPALIVE, but this can also take some time to timeout (and may have bad effects on slow connection or those with outages of interim systems). I know that I wanted to implement the capability to activate KEEPALIVE, but I am not sure if I found time to actually do it. Will let you know once I can check that.
Rainer > -----Original Message----- > From: [email protected] [mailto:rsyslog- > [email protected]] On Behalf Of Kenneth Marshall > Sent: Monday, January 11, 2010 2:52 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5 > machine > > It does seem to act like the RELP problem, but my use is only > with a regular TCP connection using @@logmachine. It had the > same symptom and restarting rsyslog cleared it up. > > Regards, > Ken > > On Mon, Jan 11, 2010 at 12:15:55PM +0100, Rainer Gerhards wrote: > > I think there is a patch (or a recommendation) regarding RELP in my > mail > > backlog. If I got it right, RELP does not necessarily detect a broken > > connection, and thus no recovery action is initiated. I'll try to get > to this > > ASAP, but I am now the second day in office and there is still a pile > of > > things I need to look into ... > > > > Rainer > > > > > -----Original Message----- > > > From: [email protected] [mailto:rsyslog- > > > [email protected]] On Behalf Of [email protected] > > > Sent: Tuesday, January 05, 2010 10:13 PM > > > To: rsyslog-users > > > Cc: [email protected] > > > Subject: Re: [rsyslog] rsyslog hang with imklog (/dev/log) on a > RHEL5 > > > machine > > > > > > this sounds like rsyslog is failing to send the logs out to the > RELP > > > server, and so is building up a large queue. restarting rsyslog > would > > > clear the queued up log messages and make it fast again. > > > > > > David Lang > > > > > > > > > On Tue, 5 Jan 2010, Kenneth Marshall wrote: > > > > > > > Date: Tue, 5 Jan 2010 13:53:49 -0600 > > > > From: Kenneth Marshall <[email protected]> > > > > Reply-To: rsyslog-users <[email protected]> > > > > To: [email protected] > > > > Cc: [email protected] > > > > Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5 > > > machine > > > > > > > > I am running rsyslog version 4.2.0 on a Redhat 5 machine > > > > and noticed slow logins to the box. The strace on the login > > > > sshd shows the following: > > > > > > > > 9937 0.000045 socket(PF_FILE, SOCK_DGRAM, 0) = 4 > > > > 9937 0.000025 fcntl(4, F_SETFD, FD_CLOEXEC) = 0 > > > > 9937 0.000019 connect(4, {sa_family=AF_FILE, > > > path="/dev/log"...}, 110) = 0 > > > > 9937 0.000040 sendto(4, "<86>Jan 5 13:36:12 sshd[9937]: > "..., > > > 90, MSG_NOSIGNAL, NULL, 0) = ? ERESTARTSYS (To be restarted) > > > > 9937 0.000042 --- SIGCHLD (Child exited) @ 0 (0) --- > > > > 9937 0.000018 sendto(4, "<86>Jan 5 13:36:12 sshd[9937]: > "..., > > > 90, MSG_NOSIGNAL, NULL, 0 <unfinished ...> > > > > 5095 7.001495 <... select resumed> ) = ? ERESTARTNOHAND (To > be > > > restarted) > > > > 5095 0.000040 --- SIGCHLD (Child exited) @ 0 (0) --- > > > > 5095 0.000025 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == > > > 0}], WNOHANG, NULL) = 9844 > > > > 5095 0.000055 wait4(-1, 0x7fffbf6d198c, WNOHANG, NULL) = 0 > > > > 5095 0.000021 rt_sigaction(SIGCHLD, NULL, {0x2ad5c3ab2740, > [], > > > SA_RESTORER, 0x2ad5c65922d0}, 8) = 0 > > > > 5095 0.000028 rt_sigreturn(0x11) = -1 EINTR (Interrupted > > > system call) > > > > 5095 0.000027 select(7, [3 5], NULL, NULL, NULL <unfinished > > > ...> > > > > 9937 8.001608 <... sendto resumed> ) = 90 > > > > 9937 0.000028 close(4) = 0 > > > > 9937 0.000039 read(6, "\0\0\5\36", 4) = 4 > > > > 9937 0.000037 read(6, "\31\0\0\0\24'\363w{\376B\364Ye > > > !\365\232\216\220\352\343\"\262\334\0\0\0\20\0\0\0"..., 1310) = > 1310 > > > > 9937 0.000104 close(6) = 0 > > > > 9937 0.000029 mmap(NULL, 1310720, PROT_READ|PROT_WRITE, > > > MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x2ad8627d9000 > > > > 9937 0.000074 munmap(0x2ad85caed000, 65536) = 0 > > > > 9937 0.000037 wait4(9938, [{WIFEXITED(s) && WEXITSTATUS(s) > == > > > 0}], 0, NULL) = 9938 > > > > 9937 0.000032 alarm(0) = 102 > > > > 9937 0.000023 rt_sigaction(SIGALRM, NULL, {0x2ad85c8637a0, > [], > > > SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, 8) = 0 > > > > 9937 0.000029 rt_sigaction(SIGALRM, {SIG_DFL, [], > > > SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, NULL, 8) = 0 > > > > ... > > > > > > > > The problem seems to be caused by writing to /dev/log which > should > > > > be being managed by the rsyslog program. I see a similar problem > > > > reported earlier on the forum: > > > > > > > > rsyslog hangs with imklog + omrelp (Same bug a imuxlog FC ?) > > > > > > > > This was for version 3.18.4 but the symptom sounded very similar. > > > > I restarted the rsyslog process and the login times returned to > > > normal. > > > > Let me know if there is something further I can do to help you > debug > > > > this matter. > > > > > > > > Regards, > > > > Ken > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
