This is useful. Thank you. Each host should have a different rate limit under the new system (I turned it back off last night, which is why everyone got everything).
Right now, the rate limit is “per IP”. So, if you have several Hosts behind a NAT, so you’ll get blocked. The new system, you can have as many hosts behind the same NAT as long as they aren’t using the same config file. A new problem being, I am seeing a ton of hosts on Amazon or Microsoft’s azure that are using the same config, so that’s a new hurdle that those people will have to overcome. I am sure there are new problems that we’ll encounter during this transition. — Sent from my iPhone > On Sep 5, 2021, at 09:09, clamav.mbou...@spamgourmet.com wrote: > > Joel Esler clamav-users@lists.clamav.net wrote: >> We are experimenting with a feature that we’ve been working with Cloudflare >> on, trying to isolate violators on a per host basis for the newest versions >> of ClamAV, instead of IP. > > I'm guessing you probably already have all the info you need but, in case it > happens to be any help, this is what I have in my freshclam logs (on a home > desktop PC, so it's not running 24-7)... > > Last messages from Friday: >> Fri Sep 3 22:13:18 2021 -> Received signal: wake up >> Fri Sep 3 22:13:18 2021 -> ClamAV update process started at Fri Sep 3 >> 22:13:18 2021 >> Fri Sep 3 22:13:18 2021 -> WARNING: Your ClamAV installation is OUTDATED! >> Fri Sep 3 22:13:18 2021 -> WARNING: Local version: 0.103.2 Recommended >> version: 0.103.3 >> Fri Sep 3 22:13:18 2021 -> DON'T PANIC! Read >> https://www.clamav.net/documents/upgrading-clamav >> Fri Sep 3 22:13:18 2021 -> daily.cld database is up-to-date (version: >> 26283, sigs: 1970262, f-level: 90, builder: ray >> nman) >> Fri Sep 3 22:13:18 2021 -> main.cvd database is up-to-date (version: 61, >> sigs: 6607162, f-level: 90, builder: sigmgr) >> Fri Sep 3 22:13:18 2021 -> bytecode.cvd database is up-to-date (version: >> 333, sigs: 92, f-level: 63, builder: awillia >> 2) >> Fri Sep 3 22:13:18 2021 -> -------------------------------------- >> Fri Sep 3 23:06:44 2021 -> Update process terminated > > So all was up-to-date then. Version 0.103.2 is the latest in the Ubuntu > 20.04 repositories, which is why I'm on that version, hence the warning. > > First messages from Saturday: >> Sat Sep 4 11:54:21 2021 -> -------------------------------------- >> Sat Sep 4 11:54:21 2021 -> freshclam daemon 0.103.2 (OS: linux-gnu, ARCH: >> x86_64, CPU: x86_64) >> Sat Sep 4 11:54:21 2021 -> ClamAV update process started at Sat Sep 4 >> 11:54:21 2021 >> Sat Sep 4 11:54:21 2021 -> WARNING: Your ClamAV installation is OUTDATED! >> Sat Sep 4 11:54:21 2021 -> WARNING: Local version: 0.103.2 Recommended >> version: 0.103.3 >> Sat Sep 4 11:54:21 2021 -> DON'T PANIC! Read >> https://www.clamav.net/documents/upgrading-clamav >> Sat Sep 4 11:54:21 2021 -> daily database available for update (local >> version: 26283, remote version: 26284) >> Sat Sep 4 11:54:23 2021 -> WARNING: downloadPatch: Can't download >> daily-26284.cdiff from https://database.clamav.net/daily-26284.cdiff >> Sat Sep 4 11:54:23 2021 -> The database server doesn't have the latest >> patch for the daily database (version 26284). The server will likely have >> updated if you check again in a few hours. >> Sat Sep 4 11:54:23 2021 -> main.cvd database is up-to-date (version: 61, >> sigs: 6607162, f-level: 90, builder: sigmgr) >> Sat Sep 4 11:54:23 2021 -> bytecode.cvd database is up-to-date (version: >> 333, sigs: 92, f-level: 63, builder: awillia2) >> Sat Sep 4 11:54:23 2021 -> -------------------------------------- >> Sat Sep 4 12:54:23 2021 -> Received signal: wake up >> Sat Sep 4 12:54:23 2021 -> ClamAV update process started at Sat Sep 4 >> 12:54:23 2021 >> Sat Sep 4 12:54:23 2021 -> WARNING: Your ClamAV installation is OUTDATED! >> Sat Sep 4 12:54:23 2021 -> WARNING: Local version: 0.103.2 Recommended >> version: 0.103.3 >> Sat Sep 4 12:54:23 2021 -> DON'T PANIC! Read >> https://www.clamav.net/documents/upgrading-clamav >> Sat Sep 4 12:54:23 2021 -> WARNING: FreshClam previously received error >> code 429 from the ClamAV Content Delivery Network (CDN). >> Sat Sep 4 12:54:23 2021 -> This means that you have been rate limited by >> the CDN. >> Sat Sep 4 12:54:23 2021 -> 1. Run FreshClam no more than once an hour to >> check for updates. >> Sat Sep 4 12:54:23 2021 -> FreshClam should check DNS first to see if >> an update is needed. >> Sat Sep 4 12:54:23 2021 -> 2. If you have more than 10 hosts on your >> network attempting to download, >> Sat Sep 4 12:54:23 2021 -> it is recommended that you set up a private >> mirror on your network using >> Sat Sep 4 12:54:23 2021 -> cvdupdate >> (https://pypi.org/project/cvdupdate/) to save bandwidth on the >> Sat Sep 4 12:54:23 2021 -> CDN and your own network. >> Sat Sep 4 12:54:23 2021 -> 3. Please do not open a ticket asking for an >> exemption from the rate limit, >> Sat Sep 4 12:54:23 2021 -> it will not be granted. >> Sat Sep 4 12:54:23 2021 -> WARNING: You are still on cool-down until after: >> 2021-09-04 15:54:23 > > So at 11:54 it determined that an update was available but it couldn't be > downloaded. It next checked an hour later at 12:54, and was apparently > already rate-limited by then (for 2 checks an hour apart, after none for 12 > hours). That was repeated at 13:43 and 14:54, then at 15:54: >> Sat Sep 4 15:54:23 2021 -> Received signal: wake up >> Sat Sep 4 15:54:23 2021 -> ClamAV update process started at Sat Sep 4 >> 15:54:23 2021 >> Sat Sep 4 15:54:23 2021 -> WARNING: Your ClamAV installation is OUTDATED! >> Sat Sep 4 15:54:23 2021 -> WARNING: Local version: 0.103.2 Recommended >> version: 0.103.3 >> Sat Sep 4 15:54:23 2021 -> DON'T PANIC! Read >> https://www.clamav.net/documents/upgrading-clamav >> Sat Sep 4 15:54:23 2021 -> WARNING: Cool-down expired, ok to try again. >> Sat Sep 4 15:54:23 2021 -> ERROR: Can't create mirrors.dat in >> /var/lib/clamav >> Sat Sep 4 15:54:23 2021 -> Hint: The database directory must be writable >> for UID XXX or GID YYY >> Sat Sep 4 15:54:23 2021 -> daily database available for update (local >> version: 26283, remote version: 26284) >> Sat Sep 4 15:54:24 2021 -> WARNING: downloadPatch: Can't download >> daily-26284.cdiff from https://database.clamav.net/daily-26284.cdiff >> Sat Sep 4 15:54:24 2021 -> The database server doesn't have the latest >> patch for the daily database (version 26284). The server will likely have >> updated if you check again in a few hours. >> Sat Sep 4 15:54:24 2021 -> main.cvd database is up-to-date (version: 61, >> sigs: 6607162, f-level: 90, builder: sigmgr) >> Sat Sep 4 15:54:24 2021 -> bytecode.cvd database is up-to-date (version: >> 333, sigs: 92, f-level: 63, builder: awillia2) >> Sat Sep 4 15:54:24 2021 -> -------------------------------------- > > At 16:54, 17:54 and 18:54 it was back to "FreshClam previously received error > code 429... you have been rate limited by the CDN". At 19:54 the cool-down > expired and it was able to check again - but failed again the same as above. > Then on cool-down at 20:54, 21:54 and 22:54, after which the PC was shut > down. This is the only instance of freshclam running on my home network, and > nothing else should be attempting to download the ClamAV databases (I haven't > been trying to download them manually, or running other instances of > freshclam). > > Today: >> Sun Sep 5 11:27:13 2021 -> -------------------------------------- >> Sun Sep 5 11:27:13 2021 -> freshclam daemon 0.103.2 (OS: linux-gnu, ARCH: >> x86_64, CPU: x86_64) >> Sun Sep 5 11:27:13 2021 -> ClamAV update process started at Sun Sep 5 >> 11:27:13 2021 >> Sun Sep 5 11:27:13 2021 -> WARNING: Your ClamAV installation is OUTDATED! >> Sun Sep 5 11:27:13 2021 -> WARNING: Local version: 0.103.2 Recommended >> version: 0.103.3 >> Sun Sep 5 11:27:13 2021 -> DON'T PANIC! Read >> https://www.clamav.net/documents/upgrading-clamav >> Sun Sep 5 11:27:13 2021 -> daily database available for update (local >> version: 26283, remote version: 26285) >> Sun Sep 5 11:27:15 2021 -> Testing database: >> '/var/lib/clamav/tmp.a9599a4ff7/clamav-431aa03fce17054479c616a2f44eae7b.tmp-daily.cld' >> ... >> Sun Sep 5 11:27:20 2021 -> Database test passed. >> Sun Sep 5 11:27:22 2021 -> daily.cld updated (version: 26285, sigs: >> 1970840, f-level: 90, builder: raynman) >> Sun Sep 5 11:27:22 2021 -> main.cvd database is up-to-date (version: 61, >> sigs: 6607162, f-level: 90, builder: sigmgr) >> Sun Sep 5 11:27:22 2021 -> bytecode.cvd database is up-to-date (version: >> 333, sigs: 92, f-level: 63, builder: awillia2) >> Sun Sep 5 11:27:22 2021 -> WARNING: Clamd was NOT notified: Can't connect >> to clamd through /var/run/clamav/clamd.ctl: No such file or directory >> Sun Sep 5 11:27:22 2021 -> -------------------------------------- >> Sun Sep 5 12:27:23 2021 -> Received signal: wake up >> Sun Sep 5 12:27:23 2021 -> ClamAV update process started at Sun Sep 5 >> 12:27:23 2021 >> Sun Sep 5 12:27:23 2021 -> WARNING: Your ClamAV installation is OUTDATED! >> Sun Sep 5 12:27:23 2021 -> WARNING: Local version: 0.103.2 Recommended >> version: 0.103.3 >> Sun Sep 5 12:27:23 2021 -> DON'T PANIC! Read >> https://www.clamav.net/documents/upgrading-clamav >> Sun Sep 5 12:27:23 2021 -> daily.cld database is up-to-date (version: >> 26285, sigs: 1970840, f-level: 90, builder: raynman) >> Sun Sep 5 12:27:23 2021 -> main.cvd database is up-to-date (version: 61, >> sigs: 6607162, f-level: 90, builder: sigmgr) >> Sun Sep 5 12:27:23 2021 -> bytecode.cvd database is up-to-date (version: >> 333, sigs: 92, f-level: 63, builder: awillia2) >> Sun Sep 5 12:27:23 2021 -> -------------------------------------- > > So it was able to successfully update today. > > -- > Mark. > > > _______________________________________________ > > clamav-users mailing list > clamav-users@lists.clamav.net > https://lists.clamav.net/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml _______________________________________________ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
