Errno 11 seems to be EAGAIN, more a status than a warning. The full Debug log may reveal details.
Rainer (soon offline for the night over here ;-)) Sent from phone, thus brief. Am 17.10.2017 22:36 schrieb "David Lang" <da...@lang.hm>: > you can copy the queue files somewhere else (best done with rsyslog > stopped), and then configure a copy of rsyslog,conf to not have any inputs, > but have the queue files and the rules for what to do with them. You can > then run a second copy of rsyslog (different pid file and using the second > config file) to go through the logs and deliver them. > > This doesn't loose the logs, and gives us a chance to figure out what's > wrong with individual logs (for example, you could make a second copy of > the queue files and then write a ruleset that processed this second copy > and only output the size of the log entries so you can see if you've hit a > size problem) > > I don't know that specific error, hopefuly Rainer will recognize it, he > will be waking up in 10-12 hours > > David Lang > > > On Tue, 17 Oct 2017, deoren wrote: > > Date: Tue, 17 Oct 2017 15:28:22 -0500 >> From: deoren <rsyslog-users-lists.adiscon....@whyaskwhy.org> >> Reply-To: rsyslog-users <rsyslog@lists.adiscon.com> >> To: rsyslog-users <rsyslog@lists.adiscon.com> >> Subject: [rsyslog] If messages are stuck in a queue, >> do you have any option other than nuking the queue file(s)? >> >> Refs: >> >> https://github.com/rsyslog/rsyslog/issues/1782 >> >> Scenario: >> >> * rsyslog v8.30.0 (Ubuntu PPA) >> * Ubuntu 16.04 >> * rsyslog sender setup to forward via omrelp (with a DA queue) to a >> remote receiver >> * nearly 1 GB of held message content in /var/spool/rsyslog >> >> >> There are 1272152 messages currently in the DA queue. I've had issues the >> last few weeks with messages getting hung up both in the in-memory and disk >> assisted queues. A restart of rsyslog will momentarily allow some recent >> messages to flow, interspersed with some older ones as rsyslog appears to >> make an attempt to drain the DA queue. >> >> After upgrading from v8.29.0 to 8.30.0 earlier this morning it appeared >> that the drain rate was slightly improved, but still very slow. ALL >> messages were being significantly delayed. >> >> For whatever reason I had the "bright" idea to modify the imfile >> configuration for a flat-file to use a regex to match what I considered a >> complete "message". I suspect that the result was a message larger than >> rsyslog was configured to handle. I reverted my changes and restarted >> rsyslog. >> >> The in-memory and disk assisted queues are once again not draining and >> messages are not being delivered at all. I found messages like this in >> /var/log/rsyslog.log: >> >> 2017-10-17T15:18:27.656310-05:00 redminebox rsyslogd: action >> 'ForwardToSawmill1' resumed (module 'omrelp') [v8.30.0 try >> http://www.rsyslog.com/e/2359 ] >> 2017-10-17T15:18:27.656399-05:00 redminebox rsyslogd: action >> 'ForwardToSawmill1' suspended (module 'omrelp'), retry 0. There should be >> messages before this one giving the reason for suspension. [v8.30.0 try >> http://www.rsyslog.com/e/2007 ] >> 2017-10-17T15:18:27.657484-05:00 redminebox rsyslogd: action >> 'ForwardToSawmill1' resumed (module 'omrelp') [v8.30.0 try >> http://www.rsyslog.com/e/2359 ] >> 2017-10-17T15:18:27.657567-05:00 redminebox rsyslogd: action >> 'ForwardToSawmill1' suspended (module 'omrelp'), retry 0. There should be >> messages before this one giving the reason for suspension. [v8.30.0 try >> http://www.rsyslog.com/e/2007 >> >> >> I temporarily toggled debug on demand and found messages like these in >> the debug log: >> >> 1297.521783203:ForwardToSawmill1 queue[DA]:Reg/w0: omrelp.c: >> relpTcpSend: sock 14, lenbuf 9721, send returned 9721 [errno 11] >> 1297.521789300:ForwardToSawmill1 queue[DA]:Reg/w0: omrelp.c: sendbuf NOT >> added to unacked list >> >> Does anyone know what "9721 [errno 11]" means? >> >> For whatever reason it does not appear that rsyslog is going to be able >> to deliver the messages, so I think I'm probably better off losing the last >> week's worth of data and starting with an empty queue. >> >> Is my only option to stop the daemon and manually clear the contents of >> /var/spool/rsyslog ? Is there perhaps a way of telling the rsyslog daemon >> to attempt one last delivery of the messages and then toss them if unable >> to do so? Perhaps a deliver "now" like Postfix has (i.e., postqueue -f)? >> >> Thanks in advance for your help. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> >> _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
