Re: [clamav-users] error code 429
On Sun, 5 Sep 2021 18:27:09 + "Joel Esler (jesler)" wrote: > Now? - All 3 systems updated successfully as soon as our DNS TXT test said the 26285 update was available (see below). This is again as it is almost every time since the download limiting mechanism stabilized. P.S. The DNS TXT record test is performed once per hour (via cron) at a different minute for each system, so as to spread out the load originating from our NAT-shared (DHCP-allocated) external IP address. = Freshclam log excerpt from one of 3 systems: -- Sunday 05 September 2021 at 05:05:02 -- /opt/clamav/bin/testclam-dns --> UPD D 26285/26284 M 61/61 B 333/333 /opt/clamav/bin/freshclam--stdout --on-update-execute=EXIT_1 ClamAV update process started at Sun Sep 5 05:05:05 2021 daily database available for update (local version: 26284, remote version: 26285) Testing database: '/opt/clamav.d/clamav.0.103.3/share/clamav/tmp.483d8f034e/clamav-121d7a6b001da0b17cc1d5ec5c709f57.tmp-daily.cld' ... Database test passed. daily.cld updated (version: 26285, sigs: 1970840, f-level: 90, builder: raynman) main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, builder: sigmgr) bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2) -- Sunday 05 September 2021 at 05:05:11 -- ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] error code 429
Now? — Sent from my iPad > On Sep 5, 2021, at 12:51, Paul Kosinski wrote: > > On Sun, 5 Sep 2021 02:45:25 + > "Joel Esler \(jesler\) via clamav-users" > wrote: > >> We are experimenting with a feature that we’ve been working with Cloudflare >> on, trying to isolate violators on a per host basis for the newest versions >> of ClamAV, instead of IP. > > - > > Maybe what we have seen today is the old problem that the "BOS" mirror is > wy behind again? We finally got the 26284 update about 17 hours later > than the TXT record claimed it was available. > > Is it possible to find out from Cloudflare why "BOS" has this problem? Some > time ago, when we were downloading full-blown CVDs (not just CDIFFs), I was > able to use another mirror which was up to date on the same day "BOS" was > behind. Now even the small CDIFFs are behind? > > Thanks, > Paul Kosinski > > -- Saturday 04 September 2021 at 22:05:01 > -- > > /opt/clamav/bin/testclam-dns > --> UPD D 26284/26283 M 61/61 B 333/333 > > /opt/clamav/bin/freshclam--stdout --on-update-execute=EXIT_1 > ClamAV update process started at Sat Sep 4 22:05:04 2021 > daily database available for update (local version: 26283, remote version: > 26284) > WARNING: downloadPatch: Can't download daily-26284.cdiff from > https://database.clamav.net/daily-26284.cdiff > The database server doesn't have the latest patch for the daily database > (version 26284). The server will likely have updated if you check again in a > few hours. > main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, > builder: sigmgr) > bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, > builder: awillia2) > > -- Saturday 04 September 2021 at 22:05:04 > -- > > > -- Saturday 04 September 2021 at 23:05:01 > -- > > /opt/clamav/bin/testclam-dns > --> UPD D 26284/26283 M 61/61 B 333/333 > > /opt/clamav/bin/freshclam--stdout --on-update-execute=EXIT_1 > ClamAV update process started at Sat Sep 4 23:05:03 2021 > daily database available for update (local version: 26283, remote version: > 26284) > Testing database: > '/opt/clamav.d/clamav.0.103.3/share/clamav/tmp.d80b44a62a/clamav-8a28103bbb9da5d3289b9e7252001905.tmp-daily.cld' > ... > Database test passed. > daily.cld updated (version: 26284, sigs: 1970546, f-level: 90, builder: > raynman) > main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, > builder: sigmgr) > bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, > builder: awillia2) > > -- Saturday 04 September 2021 at 23:05:10 > -- > > > smime.p7s Description: S/MIME cryptographic signature ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] error code 429
On Sun, 5 Sep 2021 02:45:25 + "Joel Esler \(jesler\) via clamav-users" wrote: > We are experimenting with a feature that we’ve been working with Cloudflare > on, trying to isolate violators on a per host basis for the newest versions > of ClamAV, instead of IP. - Maybe what we have seen today is the old problem that the "BOS" mirror is wy behind again? We finally got the 26284 update about 17 hours later than the TXT record claimed it was available. Is it possible to find out from Cloudflare why "BOS" has this problem? Some time ago, when we were downloading full-blown CVDs (not just CDIFFs), I was able to use another mirror which was up to date on the same day "BOS" was behind. Now even the small CDIFFs are behind? Thanks, Paul Kosinski -- Saturday 04 September 2021 at 22:05:01 -- /opt/clamav/bin/testclam-dns --> UPD D 26284/26283 M 61/61 B 333/333 /opt/clamav/bin/freshclam--stdout --on-update-execute=EXIT_1 ClamAV update process started at Sat Sep 4 22:05:04 2021 daily database available for update (local version: 26283, remote version: 26284) WARNING: downloadPatch: Can't download daily-26284.cdiff from https://database.clamav.net/daily-26284.cdiff The database server doesn't have the latest patch for the daily database (version 26284). The server will likely have updated if you check again in a few hours. main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, builder: sigmgr) bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2) -- Saturday 04 September 2021 at 22:05:04 -- -- Saturday 04 September 2021 at 23:05:01 -- /opt/clamav/bin/testclam-dns --> UPD D 26284/26283 M 61/61 B 333/333 /opt/clamav/bin/freshclam--stdout --on-update-execute=EXIT_1 ClamAV update process started at Sat Sep 4 23:05:03 2021 daily database available for update (local version: 26283, remote version: 26284) Testing database: '/opt/clamav.d/clamav.0.103.3/share/clamav/tmp.d80b44a62a/clamav-8a28103bbb9da5d3289b9e7252001905.tmp-daily.cld' ... Database test passed. daily.cld updated (version: 26284, sigs: 1970546, f-level: 90, builder: raynman) main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, builder: sigmgr) bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2) -- Saturday 04 September 2021 at 23:05:10 -- ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] Clamav download problems
Maybe I’ll ask you to experiment with me when I try again? Have you modify some things? — Sent from my iPhone > On Sep 5, 2021, at 12:12, Paul Netpresto wrote: > > Hi Joel > > I have 4 hosts each on a unique IP in the net 212.84.90.0/25. They all run > the command "/usr/bin/freshclam --quiet --on-update-execute=EXIT_1 " once > per hour. > > As far as I am aware this is within limits. > > So why did all 4 of my systems report the same issue for most of yesterday > and the first few hours of today that being. > > ClamAV update process started at Sat Sep 4 09:53:55 2021 > daily database available for update (local version: 26283, remote version: > 26284 > ) > WARNING: downloadPatch: Can't download daily-26284.cdiff from > https://database.c > lamav.net/daily-26284.cdiff > The database server doesn't have the latest patch for the daily database > (versio > n 26284). The server will likely have updated if you check again in a few > hours. > main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, > builde > r: sigmgr) > bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, > builde > r: awillia2) > > Regards Paul > >> On 05/09/2021 16:08, Joel Esler (jesler) via clamav-users wrote: >> This is useful. Thank you. >> >> Each host should have a different rate limit under the new system (I turned >> it back off last night, which is why everyone got everything). >> >> Right now, the rate limit is “per IP”. So, if you have several >> Hosts behind a NAT, so you’ll get blocked. The new system, you can have as >> many hosts behind the same NAT as long as they aren’t using the same config >> file. >> >> A new problem being, I am seeing a ton of hosts on Amazon or Microsoft’s >> azure that are using the same config, so that’s a new hurdle that those >> people will have to overcome. I am sure there are new problems that we’ll >> encounter during this transition. >> >> >> >> — >> Sent from my iPhone >> On Sep 5, 2021, at 09:09, clamav.mbou...@spamgourmet.com wrote: >>> >>> Joel Esler clamav-users@lists.clamav.net wrote: We are experimenting with a feature that we’ve been working with Cloudflare on, trying to isolate violators on a per host basis for the newest versions of ClamAV, instead of IP. >>> I'm guessing you probably already have all the info you need but, in case >>> it happens to be any help, this is what I have in my freshclam logs (on a >>> home desktop PC, so it's not running 24-7)... >>> >>> Last messages from Friday: Fri Sep 3 22:13:18 2021 -> Received signal: wake up Fri Sep 3 22:13:18 2021 -> ClamAV update process started at Fri Sep 3 22:13:18 2021 Fri Sep 3 22:13:18 2021 -> WARNING: Your ClamAV installation is OUTDATED! Fri Sep 3 22:13:18 2021 -> WARNING: Local version: 0.103.2 Recommended version: 0.103.3 Fri Sep 3 22:13:18 2021 -> DON'T PANIC! Read https://www.clamav.net/documents/upgrading-clamav Fri Sep 3 22:13:18 2021 -> daily.cld database is up-to-date (version: 26283, sigs: 1970262, f-level: 90, builder: ray nman) Fri Sep 3 22:13:18 2021 -> main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, builder: sigmgr) Fri Sep 3 22:13:18 2021 -> bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia 2) Fri Sep 3 22:13:18 2021 -> -- Fri Sep 3 23:06:44 2021 -> Update process terminated >>> So all was up-to-date then. Version 0.103.2 is the latest in the Ubuntu >>> 20.04 repositories, which is why I'm on that version, hence the warning. >>> >>> First messages from Saturday: Sat Sep 4 11:54:21 2021 -> -- Sat Sep 4 11:54:21 2021 -> freshclam daemon 0.103.2 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64) Sat Sep 4 11:54:21 2021 -> ClamAV update process started at Sat Sep 4 11:54:21 2021 Sat Sep 4 11:54:21 2021 -> WARNING: Your ClamAV installation is OUTDATED! Sat Sep 4 11:54:21 2021 -> WARNING: Local version: 0.103.2 Recommended version: 0.103.3 Sat Sep 4 11:54:21 2021 -> DON'T PANIC! Read https://www.clamav.net/documents/upgrading-clamav Sat Sep 4 11:54:21 2021 -> daily database available for update (local version: 26283, remote version: 26284) Sat Sep 4 11:54:23 2021 -> WARNING: downloadPatch: Can't download daily-26284.cdiff from https://database.clamav.net/daily-26284.cdiff Sat Sep 4 11:54:23 2021 -> The database server doesn't have the latest patch for the daily database (version 26284). The server will likely have updated if you check again in a few hours. Sat Sep 4 11:54:23 2021 -> main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, builder: sigmgr) Sat Sep 4 11:54:23 2021 -> bytecode.cvd database is up-to-date (version: 333,
Re: [clamav-users] error code 429
Thank you for being patient while I try some different things to find the middle ground. — Sent from my iPhone > On Sep 5, 2021, at 12:16, clamav.mbou...@spamgourmet.com wrote: > > No problem; good to know it was useful. > > In my case, only the one host behind the NAT (physical PC on a home broadband > connection) is running freshclam anyway, but it appears I was still being > blocked by the rate-limiting. As I understand it, that shouldn't usually > have happened even with the per-IP system. Not sure if that's an issue with > how the new system differentiates between hosts, or perhaps when the download > failed (for whatever reason) freshclam was trying several times and getting > blocked. > > I'm running Linux Mint 20, which is based on Ubuntu 20.04 and uses a lot of > packages from the Ubuntu repositories (upgraded not long after my posts here > a few months ago when I had problems with the default receive timeout in > Ubuntu 16/18.04's packages). ClamAV and freshclam are installed from the > Ubuntu 20.04 repositories, and I haven't yet needed to change the > configuration from the default - so my config will be the same as anyone else > who's installed from the Ubuntu 20.04 repo will have by default. Not sure > whether the new system would have treated everyone with this default config > as the same host, though I'd have thought IP would still be taken into > account as well. > > I'm not complaining - you've clearly had a lot of problems with the CDN being > abused (intentionally or otherwise) and need to try these things. Just trying > to give you whatever information might be useful :) > > Thanks, > Mark. > > > Joel Esler jesler via clamav-users - clamav-users@lists.clamav.net wrote: >> This is useful. Thank you. >> Each host should have a different rate limit under the new system (I turned >> it back off last night, which is why everyone got everything). >> Right now, the rate limit is “per IP”. So, if you have several >> Hosts behind a NAT, so you’ll get blocked. The new system, you can have as >> many hosts behind the same NAT as long as they aren’t using the same config >> file. >> A new problem being, I am seeing a ton of hosts on Amazon or Microsoft’s >> azure that are using the same config, so that’s a new hurdle that those >> people will have to overcome. I am sure there are new problems that we’ll >> encounter during this transition. >> — >> Sent from my iPhone On Sep 5, 2021, at 09:09, clamav.mbou...@spamgourmet.com wrote: >>> >>> Joel Esler clamav-users@lists.clamav.net wrote: We are experimenting with a feature that we’ve been working with Cloudflare on, trying to isolate violators on a per host basis for the newest versions of ClamAV, instead of IP. >>> >>> I'm guessing you probably already have all the info you need but, in case >>> it happens to be any help, this is what I have in my freshclam logs (on a >>> home desktop PC, so it's not running 24-7)... >>> >>> Last messages from Friday: Fri Sep 3 22:13:18 2021 -> Received signal: wake up Fri Sep 3 22:13:18 2021 -> ClamAV update process started at Fri Sep 3 22:13:18 2021 Fri Sep 3 22:13:18 2021 -> WARNING: Your ClamAV installation is OUTDATED! Fri Sep 3 22:13:18 2021 -> WARNING: Local version: 0.103.2 Recommended version: 0.103.3 Fri Sep 3 22:13:18 2021 -> DON'T PANIC! Read https://www.clamav.net/documents/upgrading-clamav Fri Sep 3 22:13:18 2021 -> daily.cld database is up-to-date (version: 26283, sigs: 1970262, f-level: 90, builder: ray nman) Fri Sep 3 22:13:18 2021 -> main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, builder: sigmgr) Fri Sep 3 22:13:18 2021 -> bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia 2) Fri Sep 3 22:13:18 2021 -> -- Fri Sep 3 23:06:44 2021 -> Update process terminated >>> >>> So all was up-to-date then. Version 0.103.2 is the latest in the Ubuntu >>> 20.04 repositories, which is why I'm on that version, hence the warning. >>> >>> First messages from Saturday: Sat Sep 4 11:54:21 2021 -> -- Sat Sep 4 11:54:21 2021 -> freshclam daemon 0.103.2 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64) Sat Sep 4 11:54:21 2021 -> ClamAV update process started at Sat Sep 4 11:54:21 2021 Sat Sep 4 11:54:21 2021 -> WARNING: Your ClamAV installation is OUTDATED! Sat Sep 4 11:54:21 2021 -> WARNING: Local version: 0.103.2 Recommended version: 0.103.3 Sat Sep 4 11:54:21 2021 -> DON'T PANIC! Read https://www.clamav.net/documents/upgrading-clamav Sat Sep 4 11:54:21 2021 -> daily database available for update (local version: 26283, remote version: 26284) Sat Sep 4 11:54:23 2021 -> WARNING: downloadPatch: Can't download daily-26284.cdiff
Re: [clamav-users] error code 429
No problem; good to know it was useful. In my case, only the one host behind the NAT (physical PC on a home broadband connection) is running freshclam anyway, but it appears I was still being blocked by the rate-limiting. As I understand it, that shouldn't usually have happened even with the per-IP system. Not sure if that's an issue with how the new system differentiates between hosts, or perhaps when the download failed (for whatever reason) freshclam was trying several times and getting blocked. I'm running Linux Mint 20, which is based on Ubuntu 20.04 and uses a lot of packages from the Ubuntu repositories (upgraded not long after my posts here a few months ago when I had problems with the default receive timeout in Ubuntu 16/18.04's packages). ClamAV and freshclam are installed from the Ubuntu 20.04 repositories, and I haven't yet needed to change the configuration from the default - so my config will be the same as anyone else who's installed from the Ubuntu 20.04 repo will have by default. Not sure whether the new system would have treated everyone with this default config as the same host, though I'd have thought IP would still be taken into account as well. I'm not complaining - you've clearly had a lot of problems with the CDN being abused (intentionally or otherwise) and need to try these things. Just trying to give you whatever information might be useful :) Thanks, Mark. Joel Esler jesler via clamav-users - clamav-users@lists.clamav.net wrote: This is useful. Thank you. Each host should have a different rate limit under the new system (I turned it back off last night, which is why everyone got everything). Right now, the rate limit is “per IP”. So, if you have several Hosts behind a NAT, so you’ll get blocked. The new system, you can have as many hosts behind the same NAT as long as they aren’t using the same config file. A new problem being, I am seeing a ton of hosts on Amazon or Microsoft’s azure that are using the same config, so that’s a new hurdle that those people will have to overcome. I am sure there are new problems that we’ll encounter during this transition. — Sent from my iPhone On Sep 5, 2021, at 09:09, clamav.mbou...@spamgourmet.com wrote: Joel Esler clamav-users@lists.clamav.net wrote: We are experimenting with a feature that we’ve been working with Cloudflare on, trying to isolate violators on a per host basis for the newest versions of ClamAV, instead of IP. I'm guessing you probably already have all the info you need but, in case it happens to be any help, this is what I have in my freshclam logs (on a home desktop PC, so it's not running 24-7)... Last messages from Friday: Fri Sep 3 22:13:18 2021 -> Received signal: wake up Fri Sep 3 22:13:18 2021 -> ClamAV update process started at Fri Sep 3 22:13:18 2021 Fri Sep 3 22:13:18 2021 -> WARNING: Your ClamAV installation is OUTDATED! Fri Sep 3 22:13:18 2021 -> WARNING: Local version: 0.103.2 Recommended version: 0.103.3 Fri Sep 3 22:13:18 2021 -> DON'T PANIC! Read https://www.clamav.net/documents/upgrading-clamav Fri Sep 3 22:13:18 2021 -> daily.cld database is up-to-date (version: 26283, sigs: 1970262, f-level: 90, builder: ray nman) Fri Sep 3 22:13:18 2021 -> main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, builder: sigmgr) Fri Sep 3 22:13:18 2021 -> bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia 2) Fri Sep 3 22:13:18 2021 -> -- Fri Sep 3 23:06:44 2021 -> Update process terminated So all was up-to-date then. Version 0.103.2 is the latest in the Ubuntu 20.04 repositories, which is why I'm on that version, hence the warning. First messages from Saturday: Sat Sep 4 11:54:21 2021 -> -- Sat Sep 4 11:54:21 2021 -> freshclam daemon 0.103.2 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64) Sat Sep 4 11:54:21 2021 -> ClamAV update process started at Sat Sep 4 11:54:21 2021 Sat Sep 4 11:54:21 2021 -> WARNING: Your ClamAV installation is OUTDATED! Sat Sep 4 11:54:21 2021 -> WARNING: Local version: 0.103.2 Recommended version: 0.103.3 Sat Sep 4 11:54:21 2021 -> DON'T PANIC! Read https://www.clamav.net/documents/upgrading-clamav Sat Sep 4 11:54:21 2021 -> daily database available for update (local version: 26283, remote version: 26284) Sat Sep 4 11:54:23 2021 -> WARNING: downloadPatch: Can't download daily-26284.cdiff from https://database.clamav.net/daily-26284.cdiff Sat Sep 4 11:54:23 2021 -> The database server doesn't have the latest patch for the daily database (version 26284). The server will likely have updated if you check again in a few hours. Sat Sep 4 11:54:23 2021 -> main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, builder: sigmgr) Sat Sep 4 11:54:23 2021 -> bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2) Sat Sep 4
[clamav-users] Clamav download problems
Hi Joel I have 4 hosts each on a unique IP in the net 212.84.90.0/25. They all run the command "/usr/bin/freshclam --quiet --on-update-execute=EXIT_1 " once per hour. As far as I am aware this is within limits. So why did all 4 of my systems report the same issue for most of yesterday and the first few hours of today that being. ClamAV update process started at Sat Sep 4 09:53:55 2021 daily database available for update (local version: 26283, remote version: 26284 ) WARNING: downloadPatch: Can't download daily-26284.cdiff from https://database.c lamav.net/daily-26284.cdiff The database server doesn't have the latest patch for the daily database (versio n 26284). The server will likely have updated if you check again in a few hours. main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, builde r: sigmgr) bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, builde r: awillia2) Regards Paul On 05/09/2021 16:08, Joel Esler (jesler) via clamav-users wrote: This is useful. Thank you. Each host should have a different rate limit under the new system (I turned it back off last night, which is why everyone got everything). Right now, the rate limit is “per IP”. So, if you have several Hosts behind a NAT, so you’ll get blocked. The new system, you can have as many hosts behind the same NAT as long as they aren’t using the same config file. A new problem being, I am seeing a ton of hosts on Amazon or Microsoft’s azure that are using the same config, so that’s a new hurdle that those people will have to overcome. I am sure there are new problems that we’ll encounter during this transition. — Sent from my iPhone On Sep 5, 2021, at 09:09, clamav.mbou...@spamgourmet.com wrote: Joel Esler clamav-users@lists.clamav.net wrote: We are experimenting with a feature that we’ve been working with Cloudflare on, trying to isolate violators on a per host basis for the newest versions of ClamAV, instead of IP. I'm guessing you probably already have all the info you need but, in case it happens to be any help, this is what I have in my freshclam logs (on a home desktop PC, so it's not running 24-7)... Last messages from Friday: Fri Sep 3 22:13:18 2021 -> Received signal: wake up Fri Sep 3 22:13:18 2021 -> ClamAV update process started at Fri Sep 3 22:13:18 2021 Fri Sep 3 22:13:18 2021 -> WARNING: Your ClamAV installation is OUTDATED! Fri Sep 3 22:13:18 2021 -> WARNING: Local version: 0.103.2 Recommended version: 0.103.3 Fri Sep 3 22:13:18 2021 -> DON'T PANIC! Read https://www.clamav.net/documents/upgrading-clamav Fri Sep 3 22:13:18 2021 -> daily.cld database is up-to-date (version: 26283, sigs: 1970262, f-level: 90, builder: ray nman) Fri Sep 3 22:13:18 2021 -> main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, builder: sigmgr) Fri Sep 3 22:13:18 2021 -> bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia 2) Fri Sep 3 22:13:18 2021 -> -- Fri Sep 3 23:06:44 2021 -> Update process terminated So all was up-to-date then. Version 0.103.2 is the latest in the Ubuntu 20.04 repositories, which is why I'm on that version, hence the warning. First messages from Saturday: Sat Sep 4 11:54:21 2021 -> -- Sat Sep 4 11:54:21 2021 -> freshclam daemon 0.103.2 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64) Sat Sep 4 11:54:21 2021 -> ClamAV update process started at Sat Sep 4 11:54:21 2021 Sat Sep 4 11:54:21 2021 -> WARNING: Your ClamAV installation is OUTDATED! Sat Sep 4 11:54:21 2021 -> WARNING: Local version: 0.103.2 Recommended version: 0.103.3 Sat Sep 4 11:54:21 2021 -> DON'T PANIC! Read https://www.clamav.net/documents/upgrading-clamav Sat Sep 4 11:54:21 2021 -> daily database available for update (local version: 26283, remote version: 26284) Sat Sep 4 11:54:23 2021 -> WARNING: downloadPatch: Can't download daily-26284.cdiff from https://database.clamav.net/daily-26284.cdiff Sat Sep 4 11:54:23 2021 -> The database server doesn't have the latest patch for the daily database (version 26284). The server will likely have updated if you check again in a few hours. Sat Sep 4 11:54:23 2021 -> main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, builder: sigmgr) Sat Sep 4 11:54:23 2021 -> bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2) Sat Sep 4 11:54:23 2021 -> -- Sat Sep 4 12:54:23 2021 -> Received signal: wake up Sat Sep 4 12:54:23 2021 -> ClamAV update process started at Sat Sep 4 12:54:23 2021 Sat Sep 4 12:54:23 2021 -> WARNING: Your ClamAV installation is OUTDATED! Sat Sep 4 12:54:23 2021 -> WARNING: Local version: 0.103.2 Recommended version: 0.103.3 Sat Sep 4 12:54:23 2021 -> DON'T PANIC! Read https://www.clamav.net/documents/upgrading-clamav Sat Sep 4 12:54:23 2021 ->
Re: [clamav-users] error code 429
This is useful. Thank you. Each host should have a different rate limit under the new system (I turned it back off last night, which is why everyone got everything). Right now, the rate limit is “per IP”. So, if you have several Hosts behind a NAT, so you’ll get blocked. The new system, you can have as many hosts behind the same NAT as long as they aren’t using the same config file. A new problem being, I am seeing a ton of hosts on Amazon or Microsoft’s azure that are using the same config, so that’s a new hurdle that those people will have to overcome. I am sure there are new problems that we’ll encounter during this transition. — Sent from my iPhone > On Sep 5, 2021, at 09:09, clamav.mbou...@spamgourmet.com wrote: > > Joel Esler clamav-users@lists.clamav.net wrote: >> We are experimenting with a feature that we’ve been working with Cloudflare >> on, trying to isolate violators on a per host basis for the newest versions >> of ClamAV, instead of IP. > > I'm guessing you probably already have all the info you need but, in case it > happens to be any help, this is what I have in my freshclam logs (on a home > desktop PC, so it's not running 24-7)... > > Last messages from Friday: >> Fri Sep 3 22:13:18 2021 -> Received signal: wake up >> Fri Sep 3 22:13:18 2021 -> ClamAV update process started at Fri Sep 3 >> 22:13:18 2021 >> Fri Sep 3 22:13:18 2021 -> WARNING: Your ClamAV installation is OUTDATED! >> Fri Sep 3 22:13:18 2021 -> WARNING: Local version: 0.103.2 Recommended >> version: 0.103.3 >> Fri Sep 3 22:13:18 2021 -> DON'T PANIC! Read >> https://www.clamav.net/documents/upgrading-clamav >> Fri Sep 3 22:13:18 2021 -> daily.cld database is up-to-date (version: >> 26283, sigs: 1970262, f-level: 90, builder: ray >> nman) >> Fri Sep 3 22:13:18 2021 -> main.cvd database is up-to-date (version: 61, >> sigs: 6607162, f-level: 90, builder: sigmgr) >> Fri Sep 3 22:13:18 2021 -> bytecode.cvd database is up-to-date (version: >> 333, sigs: 92, f-level: 63, builder: awillia >> 2) >> Fri Sep 3 22:13:18 2021 -> -- >> Fri Sep 3 23:06:44 2021 -> Update process terminated > > So all was up-to-date then. Version 0.103.2 is the latest in the Ubuntu > 20.04 repositories, which is why I'm on that version, hence the warning. > > First messages from Saturday: >> Sat Sep 4 11:54:21 2021 -> -- >> Sat Sep 4 11:54:21 2021 -> freshclam daemon 0.103.2 (OS: linux-gnu, ARCH: >> x86_64, CPU: x86_64) >> Sat Sep 4 11:54:21 2021 -> ClamAV update process started at Sat Sep 4 >> 11:54:21 2021 >> Sat Sep 4 11:54:21 2021 -> WARNING: Your ClamAV installation is OUTDATED! >> Sat Sep 4 11:54:21 2021 -> WARNING: Local version: 0.103.2 Recommended >> version: 0.103.3 >> Sat Sep 4 11:54:21 2021 -> DON'T PANIC! Read >> https://www.clamav.net/documents/upgrading-clamav >> Sat Sep 4 11:54:21 2021 -> daily database available for update (local >> version: 26283, remote version: 26284) >> Sat Sep 4 11:54:23 2021 -> WARNING: downloadPatch: Can't download >> daily-26284.cdiff from https://database.clamav.net/daily-26284.cdiff >> Sat Sep 4 11:54:23 2021 -> The database server doesn't have the latest >> patch for the daily database (version 26284). The server will likely have >> updated if you check again in a few hours. >> Sat Sep 4 11:54:23 2021 -> main.cvd database is up-to-date (version: 61, >> sigs: 6607162, f-level: 90, builder: sigmgr) >> Sat Sep 4 11:54:23 2021 -> bytecode.cvd database is up-to-date (version: >> 333, sigs: 92, f-level: 63, builder: awillia2) >> Sat Sep 4 11:54:23 2021 -> -- >> Sat Sep 4 12:54:23 2021 -> Received signal: wake up >> Sat Sep 4 12:54:23 2021 -> ClamAV update process started at Sat Sep 4 >> 12:54:23 2021 >> Sat Sep 4 12:54:23 2021 -> WARNING: Your ClamAV installation is OUTDATED! >> Sat Sep 4 12:54:23 2021 -> WARNING: Local version: 0.103.2 Recommended >> version: 0.103.3 >> Sat Sep 4 12:54:23 2021 -> DON'T PANIC! Read >> https://www.clamav.net/documents/upgrading-clamav >> Sat Sep 4 12:54:23 2021 -> WARNING: FreshClam previously received error >> code 429 from the ClamAV Content Delivery Network (CDN). >> Sat Sep 4 12:54:23 2021 -> This means that you have been rate limited by >> the CDN. >> Sat Sep 4 12:54:23 2021 -> 1. Run FreshClam no more than once an hour to >> check for updates. >> Sat Sep 4 12:54:23 2021 -> FreshClam should check DNS first to see if >> an update is needed. >> Sat Sep 4 12:54:23 2021 -> 2. If you have more than 10 hosts on your >> network attempting to download, >> Sat Sep 4 12:54:23 2021 -> it is recommended that you set up a private >> mirror on your network using >> Sat Sep 4 12:54:23 2021 -> cvdupdate >> (https://pypi.org/project/cvdupdate/) to save bandwidth on the >> Sat Sep 4 12:54:23 2021 -> CDN and your own network. >> Sat Sep 4 12:54:23 2021 -> 3. Please do not open
Re: [clamav-users] error code 429
Joel Esler clamav-users@lists.clamav.net wrote: We are experimenting with a feature that we’ve been working with Cloudflare on, trying to isolate violators on a per host basis for the newest versions of ClamAV, instead of IP. I'm guessing you probably already have all the info you need but, in case it happens to be any help, this is what I have in my freshclam logs (on a home desktop PC, so it's not running 24-7)... Last messages from Friday: Fri Sep 3 22:13:18 2021 -> Received signal: wake up Fri Sep 3 22:13:18 2021 -> ClamAV update process started at Fri Sep 3 22:13:18 2021 Fri Sep 3 22:13:18 2021 -> WARNING: Your ClamAV installation is OUTDATED! Fri Sep 3 22:13:18 2021 -> WARNING: Local version: 0.103.2 Recommended version: 0.103.3 Fri Sep 3 22:13:18 2021 -> DON'T PANIC! Read https://www.clamav.net/documents/upgrading-clamav Fri Sep 3 22:13:18 2021 -> daily.cld database is up-to-date (version: 26283, sigs: 1970262, f-level: 90, builder: ray nman) Fri Sep 3 22:13:18 2021 -> main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, builder: sigmgr) Fri Sep 3 22:13:18 2021 -> bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia 2) Fri Sep 3 22:13:18 2021 -> -- Fri Sep 3 23:06:44 2021 -> Update process terminated So all was up-to-date then. Version 0.103.2 is the latest in the Ubuntu 20.04 repositories, which is why I'm on that version, hence the warning. First messages from Saturday: Sat Sep 4 11:54:21 2021 -> -- Sat Sep 4 11:54:21 2021 -> freshclam daemon 0.103.2 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64) Sat Sep 4 11:54:21 2021 -> ClamAV update process started at Sat Sep 4 11:54:21 2021 Sat Sep 4 11:54:21 2021 -> WARNING: Your ClamAV installation is OUTDATED! Sat Sep 4 11:54:21 2021 -> WARNING: Local version: 0.103.2 Recommended version: 0.103.3 Sat Sep 4 11:54:21 2021 -> DON'T PANIC! Read https://www.clamav.net/documents/upgrading-clamav Sat Sep 4 11:54:21 2021 -> daily database available for update (local version: 26283, remote version: 26284) Sat Sep 4 11:54:23 2021 -> WARNING: downloadPatch: Can't download daily-26284.cdiff from https://database.clamav.net/daily-26284.cdiff Sat Sep 4 11:54:23 2021 -> The database server doesn't have the latest patch for the daily database (version 26284). The server will likely have updated if you check again in a few hours. Sat Sep 4 11:54:23 2021 -> main.cvd database is up-to-date (version: 61, sigs: 6607162, f-level: 90, builder: sigmgr) Sat Sep 4 11:54:23 2021 -> bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2) Sat Sep 4 11:54:23 2021 -> -- Sat Sep 4 12:54:23 2021 -> Received signal: wake up Sat Sep 4 12:54:23 2021 -> ClamAV update process started at Sat Sep 4 12:54:23 2021 Sat Sep 4 12:54:23 2021 -> WARNING: Your ClamAV installation is OUTDATED! Sat Sep 4 12:54:23 2021 -> WARNING: Local version: 0.103.2 Recommended version: 0.103.3 Sat Sep 4 12:54:23 2021 -> DON'T PANIC! Read https://www.clamav.net/documents/upgrading-clamav Sat Sep 4 12:54:23 2021 -> WARNING: FreshClam previously received error code 429 from the ClamAV Content Delivery Network (CDN). Sat Sep 4 12:54:23 2021 -> This means that you have been rate limited by the CDN. Sat Sep 4 12:54:23 2021 -> 1. Run FreshClam no more than once an hour to check for updates. Sat Sep 4 12:54:23 2021 -> FreshClam should check DNS first to see if an update is needed. Sat Sep 4 12:54:23 2021 -> 2. If you have more than 10 hosts on your network attempting to download, Sat Sep 4 12:54:23 2021 -> it is recommended that you set up a private mirror on your network using Sat Sep 4 12:54:23 2021 -> cvdupdate (https://pypi.org/project/cvdupdate/) to save bandwidth on the Sat Sep 4 12:54:23 2021 -> CDN and your own network. Sat Sep 4 12:54:23 2021 -> 3. Please do not open a ticket asking for an exemption from the rate limit, Sat Sep 4 12:54:23 2021 -> it will not be granted. Sat Sep 4 12:54:23 2021 -> WARNING: You are still on cool-down until after: 2021-09-04 15:54:23 So at 11:54 it determined that an update was available but it couldn't be downloaded. It next checked an hour later at 12:54, and was apparently already rate-limited by then (for 2 checks an hour apart, after none for 12 hours). That was repeated at 13:43 and 14:54, then at 15:54: Sat Sep 4 15:54:23 2021 -> Received signal: wake up Sat Sep 4 15:54:23 2021 -> ClamAV update process started at Sat Sep 4 15:54:23 2021 Sat Sep 4 15:54:23 2021 -> WARNING: Your ClamAV installation is OUTDATED! Sat Sep 4 15:54:23 2021 -> WARNING: Local version: 0.103.2 Recommended version: 0.103.3 Sat Sep 4 15:54:23 2021 -> DON'T PANIC! Read https://www.clamav.net/documents/upgrading-clamav Sat Sep 4 15:54:23 2021 -> WARNING: Cool-down expired, ok to try
Re: [clamav-users] error code 429
Hi Do I have reason to be concerned that my systems could not download yesterday's daily cdiff until the early hours of today. They are all 0.103.(2|3) release.version The experiment did not appear to impact many folk Regards Paul On 05/09/2021 03:45, Joel Esler (jesler) via clamav-users wrote: We are experimenting with a feature that we’ve been working with Cloudflare on, trying to isolate violators on a per host basis for the newest versions of ClamAV, instead of IP. — Sent from my iPhone On Sep 4, 2021, at 18:52, Jim Popovitch via clamav-users wrote: On Sat, 2021-09-04 at 14:41 -0400, Paul Kosinski wrote: Not rate limited (as we only check about once per hour, from each of 3 systems), but we're not getting updates. Seeing similar here now that the (3rd) cool-down has expired. I'm starting to suspect this is a CloudFlare issue. Under the new ClamAV CDN parlance, what exactly defines "a network". Are they expecting service providers to setup clamav caches like major hosting providers do for OS updates? -Jim P. Sep 4 22:41:43 mx3 freshclam[1253]: Cool-down expired, ok to try again. Sep 4 22:41:45 mx3 freshclam[1253]: downloadPatch: Can't download daily-26284.cdiff from https://database.clamav.net/daily-26284.cdiff Sep 4 22:41:45 mx3 freshclam[1253]: The database server doesn't have the latest patch for the daily database (version 26284). The server will likely have updated if you check again in a few hours. ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] error code 429
[ Top-posting to be consistent with previous message.] I had the same problem as Jim and Paul (which resolved itself at about 03:00 UTC, after ~19 hours). I am running the 0.103.2 from Ubuntu 21.04. On Sun, 5 Sep 2021, Joel Esler (jesler) via clamav-users wrote: We are experimenting with a feature that we’ve been working with Cloudflare on, trying to isolate violators on a per host basis for the newest versions of ClamAV, instead of IP. — Sent from my iPhone On Sep 4, 2021, at 18:52, Jim Popovitch via clamav-users wrote: On Sat, 2021-09-04 at 14:41 -0400, Paul Kosinski wrote: Not rate limited (as we only check about once per hour, from each of 3 systems), but we're not getting updates. Seeing similar here now that the (3rd) cool-down has expired. I'm starting to suspect this is a CloudFlare issue. Under the new ClamAV CDN parlance, what exactly defines "a network". Are they expecting service providers to setup clamav caches like major hosting providers do for OS updates? -Jim P. Sep 4 22:41:43 mx3 freshclam[1253]: Cool-down expired, ok to try again. Sep 4 22:41:45 mx3 freshclam[1253]: downloadPatch: Can't download daily-26284.cdiff from https://database.clamav.net/daily-26284.cdiff Sep 4 22:41:45 mx3 freshclam[1253]: The database server doesn't have the latest patch for the daily database (version 26284). The server will likely have updated if you check again in a few hours. ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml -- Andrew C. Aitchison Kendal, UK and...@aitchison.me.uk ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
