https://blog.clamav.net/2020/07/freshclam-cdiffs-effect-on-bandwidth.html<https://blog.clamav.net/2020/07/freshclam-cdiffs-effect-on-bandwidth.html?m=1>
Freshclam, cdiffs and bandwidth are your friends During a recent review of file downloads from our ClamAV CDN network, we've noticed hundreds of IPs that seem to be downloading the daily.cvd and the main.cvd thousands of times a day. There are about a dozen IPs that are downloading those to files more than 40,000 times a day. This is causing us to transfer about 250TB of data a day. We would encourage any users still doing this to cease as soon as possible. Not only does it waste our bandwidth — as we have much more efficient ways of downloading the updates — but it also wastes your bandwidth, as well. Freshclam has the ability to download partial files of updates (called cdiffs). Which are smaller, more incremental updates to the database. This allows users, and us, to manage our downloads in a much more efficient manner. We often receive the complaint, "I have to download the daily.cvd and main.cvd with Python and move the updates to an off-internet system." That's fine — it's a use case we support. However, you can do the same with freshclam and the small cdiffs. Furthermore, we also only release updates once a day. Reducing the number of updates you check for (and, subsequently, download we assume through a crontab or periodic job of some type) would also alleviate this issue. We will be constantly monitoring this in hopes that people migrate to using freshclam. Over-abusers (for instance, the top 10 IPs that are downloading main.cvd 40,000 times a day), will be immediately blocked. Further abusers may also be blocked, without notice. To mitigate, please complete the following tasks: 1. Use Freshclam instead of Python or whatever downloading script you have cron'd. 2. Reduce the checks to once or twice a day. Thank you for helping keep the ClamAV network healthy. Any questions, please see us over on the ClamAV-Users list. Sent from my iPhone
_______________________________________________ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
