Vangelis Katsikaros via clamav-users wrote:
Hi Joel, thanks for the quick response. We already download once every hour (the default ubuntu 18.04 behavior). However, we are using auto scaling and we might be running a large number of EC2 instances (a few hundreds), that could try to download simultaneously. To further scale without issues it seemed quite easy (and it was easy in practice) to use a private mirror. The system conditions and the cld & inc files is the only blocker to make this work :/
It sounds like you need to add a freshclam call as one of the last bits of spinning up new VM/container instances just before actually activating one. [edit: I saw downthread you've supposedly already done this; clearly something is going wrong close by that action.] Or include a reference set of signature files and let freshclam run its updates in the VM/container as normally configured (with PrivateMirror, you should be able to get away with setting freshclam's update check interval down to 10-15 minutes - or possibly less).
On Jan 26, 2021, at 11:46 AM, Vangelis Katsikaros via clamav-users <clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>> wrote: Hi I am using Ubuntu 18.04 and recently found out that downloaind clamav files was blocked by cloudflare. Based on https://lists.clamav.net/pipermail/clamav-users/2020-April/009482.html I contacted Joel Esler (many thanks for the quick reply) and inform me that we were downloading too often.
I did the following steps: 1) Created a private mirror as described in https://www.clamav.net/documents/private-local-mirrors and mirrored all "cvd" files. 2) Cloud-init ensures that on boot the files are downloaded from the private mirror and are always available locally. Tthen cloud-init restarts the clamav daemon.
Double-check this step. If the systemd condition is failing, or clamd is failing to find the files, then the files aren't getting where they need to.
The systemd unit file has 2 ConditionPathExistsGlob that require the additional files "main.cld", "main.inc", "daily.cld", "daily.inc" to be available, for the service to start. $ cat /lib/systemd/system/clamav-daemon.service [Unit] Description=Clam AntiVirus userspace daemon Documentation=man:clamd(8) man:clamd.conf(5) https://www.clamav.net/documents/ # Check for database existence ConditionPathExistsGlob=/var/lib/clamav/main.{c[vl]d,inc} ConditionPathExistsGlob=/var/lib/clamav/daily.{c[vl]d,inc}
Unless I misread, that means "at least one of main.cvd, main.cld, or main.inc, plus at least one of daily.cvd, daily.cld, or daily.inc"; you should not need more. If you're using the stock signature files you'll usually see bytecode.(cvd|cld|inc) as well, but it's not considered critical.
I'm not sure about the .inc files (sounds a bit familiar, legacy historic files?). The .cvd files are the compressed version of the .cld files - for long-running systems using the standard incremental updates you'll usually end up with .cld files, but I expect in your case you'll have .cvd files. IIRC at one time you did actually end up with both - or all three - but checking several instances locally I only see the .cld files.
TBH, re-reading https://www.clamav.net/documents/private-local-mirrors it seems to me you'd be better off including a baseline set of .cvd or .cld files in your instance image (periodically updated to cut down the time for an instance to finish going live), and using option 1 - using a proxy server to let freshclam just do its incremental updates on your VPS/container instances without hitting the public mirror network each time.
-kgd _______________________________________________ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml