Maybe I do not need the complete log, because... > > Debug output of the thread managing this netstream when the error > > occured: > > 4442.720793561:40976b70: netstream 0x4100cab0 with new data > > 4442.720833000:40976b70: error during recv on NSD 0x413d1ae8: > > Connection reset by peer
It looks like the remote peer has closed the session. Any idea why? Rainer > > 4442.720840716:40976b70: gtlsRcv return. nsd 0x41079868, iRet -2165, > > lenRcvBuf 0, ptrRcvBuf 0 > > 4442.720848795:40976b70: Called LogError, msg: netstream session > > 0x4100cab0 will be closed due to error > > > > 4442.720863318:40976b70: MsgSetTAG in: len 14, pszBuf: rsyslogd-2165: > > 4442.720869899:40976b70: MsgSetTAG exit: pMsg->iLenTAG 14, pMsg- > > >TAG.szBuf: rsyslogd-2165: > > 4442.720884625:40976b70: main Q: entry added, size now log 6340, phys > > 6372 entries > > 4442.720893681:40976b70: main Q: EnqueueMsg advised worker start > > 4442.720920645:40976b70: --------<NSDSEL_PTCP> calling select, active > > fds (max 19): 14 15 19 > > > > Second occurence, different netstream: > > rsyslogd: netstream session 0x8640fc0 will be closed due to error > > > > 4438.033367497:40976b70: main Q: entry added, size now log 6809, phys > > 6841 entries > > 4438.033388122:40976b70: main Q: entry added, size now log 6810, phys > > 6842 entries > > 4438.033396264:40976b70: main Q: entry added, size now log 6811, phys > > 6843 entries > > 4438.033404087:40976b70: main Q: entry added, size now log 6812, phys > > 6844 entries > > 4438.033412176:40976b70: main Q: entry added, size now log 6813, phys > > 6845 entries > > 4438.033420108:40976b70: main Q: entry added, size now log 6814, phys > > 6846 entries > > 4438.033427975:40976b70: main Q: entry added, size now log 6815, phys > > 6847 entries > > 4438.033436020:40976b70: main Q: entry added, size now log 6816, phys > > 6848 entries > > 4438.033443886:40976b70: main Q: entry added, size now log 6817, phys > > 6849 entries > > 4438.033451823:40976b70: main Q: entry added, size now log 6818, phys > > 6850 entries > > 4438.033459819:40976b70: main Q: entry added, size now log 6819, phys > > 6851 entries > > 4438.033467757:40976b70: main Q: entry added, size now log 6820, phys > > 6852 entries > > 4438.033475728:40976b70: main Q: entry added, size now log 6821, phys > > 6853 entries > > 4438.033483613:40976b70: main Q: entry added, size now log 6822, phys > > 6854 entries > > 4438.033491751:40976b70: main Q: entry added, size now log 6823, phys > > 6855 entries > > 4438.033499760:40976b70: main Q: entry added, size now log 6824, phys > > 6856 entries > > 4438.033507589:40976b70: main Q: entry added, size now log 6825, phys > > 6857 entries > > 4438.033515679:40976b70: main Q: entry added, size now log 6826, phys > > 6858 entries > > 4438.033523711:40976b70: main Q: entry added, size now log 6827, phys > > 6859 entries > > 4438.033531526:40976b70: main Q: entry added, size now log 6828, phys > > 6860 entries > > 4438.033539457:40976b70: main Q: entry added, size now log 6829, phys > > 6861 entries > > 4438.033547373:40976b70: main Q: entry added, size now log 6830, phys > > 6862 entries > > 4438.033555150:40976b70: main Q: entry added, size now log 6831, phys > > 6863 entries > > 4438.033563201:40976b70: main Q: entry added, size now log 6832, phys > > 6864 entries > > 4438.033571121:40976b70: main Q: entry added, size now log 6833, phys > > 6865 entries > > 4438.033578976:40976b70: main Q: entry added, size now log 6834, phys > > 6866 entries > > 4438.033586987:40976b70: main Q: entry added, size now log 6835, phys > > 6867 entries > > 4438.033594920:40976b70: main Q: entry added, size now log 6836, phys > > 6868 entries > > 4438.033602860:40976b70: main Q: entry added, size now log 6837, phys > > 6869 entries > > 4438.033610750:40976b70: main Q: entry added, size now log 6838, phys > > 6870 entries > > 4438.033618581:40976b70: main Q: entry added, size now log 6839, phys > > 6871 entries > > 4438.033626573:40976b70: main Q: entry added, size now log 6840, phys > > 6872 entries > > 4438.033635275:40976b70: main Q: entry added, size now log 6841, phys > > 6873 entries > > 4438.033644155:40976b70: main Q: entry added, size now log 6842, phys > > 6874 entries > > 4438.033652187:40976b70: main Q: entry added, size now log 6843, phys > > 6875 entries > > 4438.033660012:40976b70: main Q: entry added, size now log 6844, phys > > 6876 entries > > 4438.033667977:40976b70: main Q: entry added, size now log 6845, phys > > 6877 entries > > 4438.033675943:40976b70: main Q: entry added, size now log 6846, phys > > 6878 entries > > 4438.033683820:40976b70: main Q: entry added, size now log 6847, phys > > 6879 entries > > 4438.033691877:40976b70: main Q: entry added, size now log 6848, phys > > 6880 entries > > 4438.033699825:40976b70: main Q: entry added, size now log 6849, phys > > 6881 entries > > 4438.033707743:40976b70: main Q: entry added, size now log 6850, phys > > 6882 entries > > 4438.033716401:40976b70: main Q: MultiEnqObj advised worker start > > 4438.033735637:40976b70: --------<NSDSEL_PTCP> calling select, active > > fds (max 19): 14 15 16 18 19 > > 4438.033772666:40976b70: netstream 0x8640fc0 with new data > > 4438.033792922:40976b70: error during recv on NSD 0x8612ca0: > > Connection reset by peer > > 4438.033800723:40976b70: gtlsRcv return. nsd 0x863fd58, iRet -2165, > > lenRcvBuf 0, ptrRcvBuf 0 > > 4438.033808563:40976b70: Called LogError, msg: netstream session > > 0x8640fc0 will be closed due to error > > > > 4438.033821043:40976b70: MsgSetTAG in: len 14, pszBuf: rsyslogd-2165: > > 4438.033827708:40976b70: MsgSetTAG exit: pMsg->iLenTAG 14, pMsg- > > >TAG.szBuf: rsyslogd-2165: > > 4438.033850214:40976b70: main Q: entry added, size now log 6851, phys > > 6883 entries > > 4438.033859170:40976b70: main Q: EnqueueMsg advised worker start > > 4438.033881307:40976b70: --------<NSDSEL_PTCP> calling select, active > > fds (max 19): 14 15 18 19 > > > > Because of the gtlsRcv I assume this happens on the outgoing > > connection since this is the only encrypted communication channel, but > > I am not sure even this assumption is correct. > > If you have anything you want me to provide or try, I'd be happy to do so. > > Many thanks in advance! > > Best regards, > > Andreas > > > > > > Rainer Gerhards <[email protected]> [08:09:11 13:11] wrote: > > > You should update to the latest stable version and see if the > > > problem persists. > > > > > > Rainer > > > > -----Original Message----- > > > > From: [email protected] [mailto:rsyslog- > > > > [email protected]] On Behalf Of Andreas Grosse > > > > Sent: Thursday, September 08, 2011 12:11 PM > > > > To: [email protected] > > > > Subject: [rsyslog] netstream errors when logging with a high > > > > message rate > > > > > > > > Hi, > > > > I am running rsyslog 5.8.3 as a central log collector which then > > > > sends the log messages to an archive using tls encryption. When > > > > the log message rate increases, I start seeing log messages like this: > > > > > > > > rsyslog: netstream session 0x8b05ef0 will be closed due to error > > > > [try > > > > http://www.rsyslog.com/e/2165 ] > > > > > > > > The logging does not stop and the tcp connection to the remote > > > > archive does not break, though - it just starts spewing out these > > > > messages up to two times per minute. > > > > I enabled additional debugging, and the logfile contained this: > > > > > > > > 9275.065615635:40976b70: netstream 0x41000c50 with new data > > > > 9275.065635173:40976b70: error during recv on NSD 0x41000b88: > > > > Connection reset by peer > > > > 9275.065641798:40976b70: gtlsRcv return. nsd 0x413fff98, iRet > > > > -2165, lenRcvBuf 0, ptrRcvBuf 0 > > > > 9275.065648064:40976b70: Called LogError, msg: netstream session > > > > 0x41000c50 will be closed due to error > > > > > > > > 9275.065675043:40976b70: main Q: entry added, size now log 6919, > > > > phys > > > > 6951 entries > > > > 9275.065682225:40976b70: main Q: EnqueueMsg advised worker start > > > > 9275.065707944:40976b70: --------<NSDSEL_PTCP> calling select, > > > > active fds (max 19): 14 15 16 19 > > > > > > > > On the receiving end I get no notification of an error happening > > > > at all. Following is the configuration the I use: > > > > > > > > $MaxMessageSize 64k > > > > $RepeatedMsgReduction off > > > > $EscapeControlCharactersOnReceive off > > > > $WorkDirectory /var/rsyslog # default location for work (spool) files > > > > > > > > $ModLoad imtcp > > > > $ModLoad imudp > > > > $ModLoad imptcp > > > > $ModLoad omuxsock > > > > $ModLoad impstats > > > > > > > > $InputPTCPServerListenIP 127.0.0.1 $InputPTCPServerRun 10100 > > > > > > > > $PStatsInterval 300 > > > > > > > > # log local syslog messages back to syslog-ng $OMUxSockSocket > > > > /dev/tosyslog if $programname startswith 'rsyslog' then :omuxsock: > > > > if $programname startswith 'rsyslog' then ~ > > > > > > > > $ActionQueueType LinkedList > > > > $ActionQueueFileName srvrfwd > > > > $ActionResumeRetryCount -1 > > > > $ActionQueueSaveOnShutdown on > > > > $ActionQueueMaxDiskSpace 819200 > > > > > > > > $DefaultNetstreamDriver gtls > > > > > > > > $DefaultNetstreamDriverCAFile /etc/ca/cacert.pem > > > > $DefaultNetstreamDriverCertFile /etc/client.pem > > > > $DefaultNetstreamDriverKeyFile /etc/client.key > > > > > > > > $ActionSendStreamDriverMode 1 > > > > $ActionSendStreamDriverAuthMode x509/certvalid > > > > > > > > $InputTCPServerStreamDriverMode 0 > > > > $InputTCPServerRun 10101 > > > > $UDPServerRun 10101 > > > > > > > > *.* @@(o,z0)loghost:5077;RSYSLOG_SyslogProtocol23Format > > > > > > > > > > > > If you need more data from the debug log, just ask. This is also > > > > easy to reproduce, therefore I am able to try some things if you > > > > come up with suggestions what happens there and how to get rid of > > > > those error messages. Thank you for your help! > > > > > > > > Best regards, > > > > Andreas Grosse > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
