It happened again this afternoon: 2022-12-15T14:01:13.006027-06:00 hermes rsyslogd[10975]: rsyslogd[internal_messages]: 793 messages lost due to rate-limiting (500 allowed within 5 seconds) 2022-12-15T14:01:19.005580-06:00 hermes rsyslogd[10975]: rsyslogd[internal_messages]: 1272 messages lost due to rate-limiting (500 allowed within 5 seconds) 2022-12-15T14:01:25.000544-06:00 hermes rsyslogd[10975]: rsyslogd[internal_messages]: 870 messages lost due to rate-limiting (500 allowed within 5 seconds) 2022-12-15T14:01:31.002353-06:00 hermes rsyslogd[10975]: rsyslogd[internal_messages]: 1041 messages lost due to rate-limiting (500 allowed within 5 seconds)
On Wed, Dec 14, 2022 at 11:31 AM Rainer Gerhards <[email protected]> wrote: > I ignore the database logging issue. When you have rate-limiting > issues again, please report, together with the description of what > happens. > > If you think this is related to mysql, please address that issue first. > > Rainer > > El mié, 14 dic 2022 a las 17:48, helices > (<[email protected]>) escribió: > > > > REF: Rsyslogd/ommysql.so: Not writing to DB intermittently > > > > Rainer asked us to start a new post for the rate-limit issue. > > > > > > A few of many hundreds of rate-limit errors and lost messages: > > > > 2022-12-13T02:23:44.003241-06:00 hermes rsyslogd[2539]: > rsyslogd[internal_messages]: 1792 messages lost due to rate-limiting (500 > allowed within 5 seconds) > > 2022-12-13T02:23:50.001278-06:00 hermes rsyslogd[2539]: > rsyslogd[internal_messages]: 1779 messages lost due to rate-limiting (500 > allowed within 5 seconds) > > 2022-12-13T02:23:56.001273-06:00 hermes rsyslogd[2539]: > rsyslogd[internal_messages]: 1835 messages lost due to rate-limiting (500 > allowed within 5 seconds) > > 2022-12-13T02:24:02.005300-06:00 hermes rsyslogd[2539]: > rsyslogd[internal_messages]: 1768 messages lost due to rate-limiting (500 > allowed within 5 seconds) > > s > > > > > > # date; grep -v "^\(#\|\s*$\)" /etc/rsyslog.conf ;date > > Wed Dec 14 10:35:41 CST 2022 > > $DebugFile /var/log/rsyslog.debug > > $DebugLevel 2 > > module(load="imjournal" Ratelimit.Burst="30000" > Ratelimit.Interval="1000" StateFile="imjournal.state") > > module(load="imklog") > > module(load="immark") > > module(load="impstats" interval="600" severity="7") > > syslog.=debug /var/log/rsyslog-stats > > module(load="imtcp") > > input(type="imtcp" port="514") > > module(load="imudp") > > input(type="imudp" port="514") > > module(load="ommysql.so") > > global(workDirectory="/var/lib/rsyslog") > > authpriv.none;cron.none;*.info;mail.none /var/log/messages > > authpriv.* /var/log/secure > > cron.* /var/log/cron > > *.emerg :omusrmsg:* > > ftp.* /var/log/vsftpd.log > > local7.* /var/log/boot.log > > mail.* /var/log/maillog > > uucp,news.crit /var/log/spooler > > $ActionName Ftp > > $ActionQueueFileName dbFtpQueue # Set file name, also enables disk mode > > $ActionQueueSaveOnShutdown on # Save messages to disk on shutdown > > $ActionQueueType LinkedList # Use asynchronous processing > > $ActionResumeRetryCount -1 # Infinite retries on insert failure > > ftp.* > :ommysql:10.199.5.177,vsftplog,hermesvsftplog,_____ > > $ActionName Sftp > > $ActionQueueFileName dbSftpQueue # Set file name, also enables disk > mode > > $ActionQueueSaveOnShutdown on # Save messages to disk on shutdown > > $ActionQueueType LinkedList # Use asynchronous processing > > $ActionResumeRetryCount -1 # Infinite retries on insert failure > > authpriv.* > :ommysql:10.199.5.177,sftplogDB,hermesvsftplog,_____ > > $ActionName Admin > > $ActionQueueFileName ZenossQueue # Set file name, also enables disk mode > > $ActionQueueSaveOnShutdown on # Save messages to disk on shutdown > > $ActionQueueType LinkedList # Use asynchronous processing > > $ActionResumeRetryCount -1 # Infinite retries on insert failure > > *.* @@10.199.1.160 > > Wed Dec 14 10:35:41 CST 2022 > > > > > > Rainer asked us to setup a debug log, according to: > > https://www.rsyslog.com/doc/master/troubleshooting/howtodebug.html > > > > Initial startup here: > > https://pastebin.com/DUgwmPC > > > > > > No rate-limiting occurred since early yesterday (12/13) morning. This > appears to be associated with the errors and multi-line syslog entries > mentioned in the other post. > > > > The sole intent of the database logging is tracking all incoming remote > file transfer (SFTP) activities. There is a firewall between this host and > the internet. Only "whitelisted" IP addresses can get through, and are to > be inserted into the database. > > > > Apparently, at least one client connects in the early morning hours, and > this unusual SFTP unusual activity results in multi-line syslog entries > that come in very large numbers. One problem is, the multiple line entries > are not written to /var/log/messages, are not inserted into the database, > and rate-limiting obscures all content. Hence, this support request is our > attempt to understand what is happening, after which we can act to correct > these problems. > > > > Interestingly, we are not aware of any missing files from this or any > other file transfer clients. > _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
