Yes - see pastebin link in 1st message.
On Thu, Dec 15, 2022, 15:43 David Lang <[email protected]> wrote: > did you post the full debug log at startup? > > since you are attempting to set the limit higher, but this is showing the > default limit, there has to be something wrong with the config or the > config > parsing. > > since the trigger is only 500 logs in 5 seconds, you should be able to use > logger to generate this many messages rather than waiting for it to happen. > > David Lang > > On Thu, 15 Dec 2022, helices wrote: > > > It happened again this afternoon: > > > > 2022-12-15T14:01:13.006027-06:00 hermes rsyslogd[10975]: > > rsyslogd[internal_messages]: 793 messages lost due to rate-limiting (500 > > allowed within 5 seconds) > > 2022-12-15T14:01:19.005580-06:00 hermes rsyslogd[10975]: > > rsyslogd[internal_messages]: 1272 messages lost due to rate-limiting (500 > > allowed within 5 seconds) > > 2022-12-15T14:01:25.000544-06:00 hermes rsyslogd[10975]: > > rsyslogd[internal_messages]: 870 messages lost due to rate-limiting (500 > > allowed within 5 seconds) > > 2022-12-15T14:01:31.002353-06:00 hermes rsyslogd[10975]: > > rsyslogd[internal_messages]: 1041 messages lost due to rate-limiting (500 > > allowed within 5 seconds) > > > > On Wed, Dec 14, 2022 at 11:31 AM Rainer Gerhards < > [email protected]> > > wrote: > > > >> I ignore the database logging issue. When you have rate-limiting > >> issues again, please report, together with the description of what > >> happens. > >> > >> If you think this is related to mysql, please address that issue first. > >> > >> Rainer > >> > >> El mié, 14 dic 2022 a las 17:48, helices > >> (<[email protected]>) escribió: > >>> > >>> REF: Rsyslogd/ommysql.so: Not writing to DB intermittently > >>> > >>> Rainer asked us to start a new post for the rate-limit issue. > >>> > >>> > >>> A few of many hundreds of rate-limit errors and lost messages: > >>> > >>> 2022-12-13T02:23:44.003241-06:00 hermes rsyslogd[2539]: > >> rsyslogd[internal_messages]: 1792 messages lost due to rate-limiting > (500 > >> allowed within 5 seconds) > >>> 2022-12-13T02:23:50.001278-06:00 hermes rsyslogd[2539]: > >> rsyslogd[internal_messages]: 1779 messages lost due to rate-limiting > (500 > >> allowed within 5 seconds) > >>> 2022-12-13T02:23:56.001273-06:00 hermes rsyslogd[2539]: > >> rsyslogd[internal_messages]: 1835 messages lost due to rate-limiting > (500 > >> allowed within 5 seconds) > >>> 2022-12-13T02:24:02.005300-06:00 hermes rsyslogd[2539]: > >> rsyslogd[internal_messages]: 1768 messages lost due to rate-limiting > (500 > >> allowed within 5 seconds) > >>> s > >>> > >>> > >>> # date; grep -v "^\(#\|\s*$\)" /etc/rsyslog.conf ;date > >>> Wed Dec 14 10:35:41 CST 2022 > >>> $DebugFile /var/log/rsyslog.debug > >>> $DebugLevel 2 > >>> module(load="imjournal" Ratelimit.Burst="30000" > >> Ratelimit.Interval="1000" StateFile="imjournal.state") > >>> module(load="imklog") > >>> module(load="immark") > >>> module(load="impstats" interval="600" severity="7") > >>> syslog.=debug /var/log/rsyslog-stats > >>> module(load="imtcp") > >>> input(type="imtcp" port="514") > >>> module(load="imudp") > >>> input(type="imudp" port="514") > >>> module(load="ommysql.so") > >>> global(workDirectory="/var/lib/rsyslog") > >>> authpriv.none;cron.none;*.info;mail.none /var/log/messages > >>> authpriv.* /var/log/secure > >>> cron.* /var/log/cron > >>> *.emerg :omusrmsg:* > >>> ftp.* /var/log/vsftpd.log > >>> local7.* /var/log/boot.log > >>> mail.* /var/log/maillog > >>> uucp,news.crit /var/log/spooler > >>> $ActionName Ftp > >>> $ActionQueueFileName dbFtpQueue # Set file name, also enables disk > mode > >>> $ActionQueueSaveOnShutdown on # Save messages to disk on shutdown > >>> $ActionQueueType LinkedList # Use asynchronous processing > >>> $ActionResumeRetryCount -1 # Infinite retries on insert failure > >>> ftp.* > >> :ommysql:10.199.5.177,vsftplog,hermesvsftplog,_____ > >>> $ActionName Sftp > >>> $ActionQueueFileName dbSftpQueue # Set file name, also enables disk > >> mode > >>> $ActionQueueSaveOnShutdown on # Save messages to disk on shutdown > >>> $ActionQueueType LinkedList # Use asynchronous processing > >>> $ActionResumeRetryCount -1 # Infinite retries on insert failure > >>> authpriv.* > >> :ommysql:10.199.5.177,sftplogDB,hermesvsftplog,_____ > >>> $ActionName Admin > >>> $ActionQueueFileName ZenossQueue # Set file name, also enables disk > mode > >>> $ActionQueueSaveOnShutdown on # Save messages to disk on shutdown > >>> $ActionQueueType LinkedList # Use asynchronous processing > >>> $ActionResumeRetryCount -1 # Infinite retries on insert failure > >>> *.* @@10.199.1.160 > >>> Wed Dec 14 10:35:41 CST 2022 > >>> > >>> > >>> Rainer asked us to setup a debug log, according to: > >>> https://www.rsyslog.com/doc/master/troubleshooting/howtodebug.html > >>> > >>> Initial startup here: > >>> https://pastebin.com/DUgwmPC > >>> > >>> > >>> No rate-limiting occurred since early yesterday (12/13) morning. This > >> appears to be associated with the errors and multi-line syslog entries > >> mentioned in the other post. > >>> > >>> The sole intent of the database logging is tracking all incoming remote > >> file transfer (SFTP) activities. There is a firewall between this host > and > >> the internet. Only "whitelisted" IP addresses can get through, and are > to > >> be inserted into the database. > >>> > >>> Apparently, at least one client connects in the early morning hours, > and > >> this unusual SFTP unusual activity results in multi-line syslog entries > >> that come in very large numbers. One problem is, the multiple line > entries > >> are not written to /var/log/messages, are not inserted into the > database, > >> and rate-limiting obscures all content. Hence, this support request is > our > >> attempt to understand what is happening, after which we can act to > correct > >> these problems. > >>> > >>> Interestingly, we are not aware of any missing files from this or any > >> other file transfer clients. > >> > > _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
