I'm not sure if the safebrowsing.cld is included in the daily cdiff, but the current safebrowsing.cld takes between 50 and 70 seconds to *load* into clamscan, where a copy from February loads in <5 seconds.
safebrowsing data: Old (fast): ClamAV-VDB:13 Feb 2019 13-16 -0500:48472:3041760:63:X:X:google:1550081775 New (slow): ClamAV-VDB:05 Mar 2019 19-20 -0500:48473:3229612:63:X:X:google:1551831615 Anyone know what might have changed in there to so drastically increased the load time? This happened after freshclam ran last night. # /opt/clamav/clamav/bin/clamscan -d ~/safebrowsing.cld samples/clam_test.html samples/clam_test.html: OK ----------- SCAN SUMMARY ----------- Known viruses: 3041760 Engine version: 0.100.2 Scanned directories: 0 Scanned files: 1 Infected files: 0 Data scanned: 0.00 MB Data read: 0.00 MB (ratio 0.00:1) Time: 2.423 sec (0 m 2 s) # /opt/clamav/clamav/bin/clamscan -d /opt/clamav/var/lib/clamav/safebrowsing.cld samples/clam_test.html samples/clam_test.html: OK ----------- SCAN SUMMARY ----------- Known viruses: 3229612 Engine version: 0.100.2 Scanned directories: 0 Scanned files: 1 Infected files: 0 Data scanned: 0.00 MB Data read: 0.00 MB (ratio 0.00:1) Time: 64.429 sec (1 m 4 s) On Wed, Mar 6, 2019 at 10:17 AM Micah Snyder (micasnyd) via clamav-users < clamav-users@lists.clamav.net> wrote: > I confirmed with our signature management team that the extended time > processing daily-25380 is because this change is significantly larger than > a standard update. > This update drops 768053 hash-based signatures for malware that is > detected by other more efficient logical signatures. The net result will > be a leaner database that should load a little faster and take up less > memory. > > The validation stage when creating the daily had estimated less than 26 > minutes for the cdiff to apply. You may be correct that it's much faster > on x86 than on Sparc. 3h15m is definitely worse than expected, and I > apologize for the inconvenience. > > Regards, > Micah > > Micah Snyder > ClamAV Development > Talos > Cisco Systems, Inc. > > > On 3/6/19, 9:31 AM, "Pierre Dehaen" <deha...@drever.be> wrote: > > Yes Micah, it finished while I was checking the computer because of > the messages received > on the mailing list. > > $ tail -50 /var/log/freshclam.log > ... > -------------------------------------- > ClamAV update process started at Wed Mar 6 11:37:46 2019 > WARNING: Your ClamAV installation is OUTDATED! > WARNING: Local version: 0.100.0 Recommended version: 0.101.1 > DON'T PANIC! Read https://www.clamav.net/documents/upgrading-clamav > securiteinfo.hdb is up to date (version: custom database) > securiteinfo.ign2 is up to date (version: custom database) > Downloading javascript.ndb [*] > javascript.ndb updated (version: custom database, sigs: 45008) > securiteinfohtml.hdb is up to date (version: custom database) > securiteinfoascii.hdb is up to date (version: custom database) > securiteinfopdf.hdb is up to date (version: custom database) > Downloading spam_marketing.ndb [*] > spam_marketing.ndb updated (version: custom database, sigs: 24199) > main.cld is up to date (version: 58, sigs: 4566249, f-level: 60, > builder: sigmgr) > Downloading daily-25380.cdiff [100%] > daily.cld updated (version: 25380, sigs: 1503528, f-level: 63, > builder: raynman) > bytecode.cld is up to date (version: 328, sigs: 94, f-level: 63, > builder: neo) > Database updated (6139078 signatures) from db.be.clamav.net (IP: > 104.16.219.84) > Clamd successfully notified about the update. > > $ ls -l /var/log/freshclam.log > -rw-r--r-- 1 clamav clamav 701634 Mar 6 14:51 > /var/log/freshclam.log > > It ran from 11:37 to 14:51. It might run faster on x86 computers > though. > > Pierre > > On 6 Mar 2019 at 14:20, Micah Snyder (micasnyd) via clamav-users wrote: > > Pierre, > > So you're saying it actually did finish after 3 hours, 15 minutes on > its own? That is good news > for all of the automated systems, even if this is a potentially > terrible bug. > > I'm still investigating the cause, and asking our signature management > team if they have any > additional details. > > Micah > > Micah Snyder > ClamAV Development > Talos > Cisco Systems, Inc. > > > > On 3/6/19, 9:06 AM, "clamav-users on behalf of Pierre Dehaen" > <clamav-users- > boun...@lists.clamav.net on behalf of deha...@drever.be> wrote: > > Here too: it took about 3 hours and 15 minutes to calm down > (SPARC, Solaris 11, > v0.100.0)... without noticiable error in freshclam.log. > > On 6 Mar 2019 at 6:27, J.R. via clamav-users wrote: > > > When crontab execs freshclam > > CPU server goes to 100% > > Hanged finishing Downloading daily-25380.cdiff [100%] > > Just checked my server and it happened to me too! A little after > 5am > central time. :( > > _______________________________________________ > > clamav-users mailing list > clamav-users@lists.clamav.net > https://lists.clamav.net/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml > > > _______________________________________________ > > clamav-users mailing list > clamav-users@lists.clamav.net > https://lists.clamav.net/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml > > > > _______________________________________________ > > clamav-users mailing list > clamav-users@lists.clamav.net > https://lists.clamav.net/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml > > > > > > _______________________________________________ > > clamav-users mailing list > clamav-users@lists.clamav.net > https://lists.clamav.net/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml >
_______________________________________________ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
