Hi all, We've been experiencing this slowdown too. We run every DB update through an extra FP test against a number of recent Mac OS installs (OS X 10.6 - 10.14, as well as some well-known 3rd party apps) just to weed out any potentially overzealous signatures. On the new Mac Minis this FP test used to take around 45 minutes to complete, it now takes almost 3 hours (176 minutes).
>From our logs, looking only at the smallest disk we check (Mac OS X 10.6.8) I've managed to compile the following list of dates, scan times and ClamAV DB version numbers. For months, the 10.6 disk used to take around 3m 20s to scan. It always jumped up and down a bit, but really hasn't been right since around the middle of February. The list is best viewed using a mono-spaced font. I've marked (with 3 asterisks) scans where the time seems to indicate an issue with the DB update. Hopefully this helps someone to narrow things down a bit. Mark dd/mm/yy duration DNS Txt 5/2/19 3m 14s TXT from DNS: 0.101.1:58:25351:1549376940:1:63:48440:328 6/2/19 3m 20s TXT from DNS: 0.101.1:58:25352:1549466941:1:63:48444:328 11/2/19 3m 20s TXT from DNS: 0.101.1:58:25356:1549837740:1:63:48460:328 11/2/19 3m 25s TXT from DNS: 0.101.1:58:25356:1549877342:1:63:48462:328 11/2/19 3m 19s TXT from DNS: 0.101.1:58:25357:1549881900:1:63:48462:328 12/2/19 3m 22s TXT from DNS: 0.101.1:58:25357:1549963741:1:63:48466:328 13/2/19 3m 22s TXT from DNS: 0.101.1:58:25358:1550050141:1:63:48470:328 14/2/19 3m 22s TXT from DNS: 0.101.1:58:25359:1550140140:1:63:48472:328 16/2/19 6m 38s TXT from DNS: 0.101.1:58:25361:1550269740:1:63:48472:328 *** 17/2/19 7m 35s TXT from DNS: 0.101.1:58:25362:1550348940:1:63:48472:328 18/2/19 7m 41s TXT from DNS: 0.101.1:58:25363:1550442540:1:63:48472:328 18/2/19 4m 22s TXT from DNS: 0.101.1:58:25364:1550492940:1:63:48472:328 19/2/19 4m 28s TXT from DNS: 0.101.1:58:25365:1550579340:1:63:48472:328 20/2/19 4m 30s TXT from DNS: 0.101.1:58:25365:1550658540:1:63:48472:328 21/2/19 4m 28s TXT from DNS: 0.101.1:58:25366:1550744940:1:63:48472:328 22/2/19 4m 36s TXT from DNS: 0.101.1:58:25368:1550842141:1:63:48472:328 24/2/19 7m 51s TXT from DNS: 0.101.1:58:25370:1551040140:1:63:48472:328 *** 25/2/19 4m 31s TXT from DNS: 0.101.1:58:25371:1551092103:1:63:48472:328 26/2/19 4m 41s TXT from DNS: 0.101.1:58:25372:1551177619:1:63:48472:328 27/2/19 4m 29s TXT from DNS: 0.101.1:58:25373:1551277740:1:63:48472:328 28/2/19 4m 28s TXT from DNS: 0.101.1:58:25373:1551349740:1:63:48472:328 1/3/19 4m 39s TXT from DNS: 0.101.1:58:25374:1551443340:1:63:48472:328 3/3/19 8m 14s TXT from DNS: 0.101.1:58:25376:1551558540:1:63:48472:328 *** 3/3/19 8m 45s TXT from DNS: 0.101.1:58:25377:1551644940:1:63:48472:328 4/3/19 4m 51s TXT from DNS: 0.101.1:58:25377:1551691742:1:63:48472:328 4/3/19 4m 52s TXT from DNS: 0.101.1:58:25378:1551709740:1:63:48472:328 5/3/19 5m 6s TXT from DNS: 0.101.1:58:25379:1551796140:1:63:48472:328 6/3/19 5m 7s TXT from DNS: 0.101.1:58:25380:1551868140:1:63:48473:328 7/3/19 5m 15s TXT from DNS: 0.101.1:58:25381:1551953509:1:63:48474:328 8/3/19 5m 14s TXT from DNS: 0.101.1:58:25382:1552048140:1:63:48478:328 9/3/19 5m 7s TXT from DNS: 0.101.1:58:25383:1552163340:1:63:48482:328 11/3/19 5m 14s TXT from DNS: 0.101.1:58:25384:1552253340:1:63:48485:328 11/3/19 5m 24s TXT from DNS: 0.101.1:58:25385:1552302125:1:63:48487:328 12/3/19 5m 42s TXT from DNS: 0.101.1:58:25386:1552388890:1:63:48490:328 13/3/19 5m 44s TXT from DNS: 0.101.1:58:25386:1552465741:1:63:48492:328 14/3/19 7m 24s TXT from DNS: 0.101.1:58:25388:1552559341:1:63:48495:328 *** 15/3/19 8m 56s TXT from DNS: 0.101.1:58:25389:1552645741:1:63:48498:328 *** 18/3/19 10m 49s TXT from DNS: 0.101.1:58:25392:1552904941:1:63:48507:328 *** 19/3/19 10m 19s TXT from DNS: 0.101.1:58:25393:1552991341:1:63:48510:328 20/3/19 10m 43s TXT from DNS: 0.101.1:58:25394:1553074140:1:63:48513:328 22/3/19 10m 58s TXT from DNS: 0.101.1:58:25395:1553180408:1:63:48517:328 22/3/19 10m 58s TXT from DNS: 0.101.1:58:25396:1553246940:1:63:48519:328 On Sat, 23 Mar 2019 at 23:26, Al Varnell via clamav-users < clamav-users@lists.clamav.net> wrote: > Sorry, I misinterpreted the meaning of "crawled" thinking it referred to > some sort of compromise of the data. > > -Al- > > On Mar 23, 2019, at 09:42, Jean-Michel via clamav-users < > clamav-users@lists.clamav.net> wrote: > > See Maarten Broekman tests above > https://lists.clamav.net/pipermail/clamav-users/2019-March/007737.html > > *De :* Al Varnell <alvarn...@mac.com> > *Envoyé :* samedi 23 mars 2019 10:55 > *À :* ClamAV users ML <clamav-users@lists.clamav.net> > *Objet :* Re: [clamav-users] Scan very slow > > Reference? First I'm hearing of any such thing. > > -Al- > > > On Mar 23, 2019, at 02:26, Jean-Michel via clamav-users < > clamav-users@lists.clamav.net> wrote: > > Hi, > Micah Snyder, Do you know if Clamav was able to trace the orgine of > getting crawled in the database "daily.cld" and was able to fix the problem? > Regards > > *De :* Micah Snyder (micasnyd) <micas...@cisco.com> > *Envoyé :* lundi 18 mars 2019 18:09 > *À :* ClamAV users ML <clamav-users@lists.clamav.net> > *Objet :* Re: [clamav-users] Scan very slow > > Maarten, > > This is very concerning, and the details you’ve provide are quite > helpful. Thank you for investigating. > Hopefully we can figure out why the newer daily.cld/cvd is scanning > significantly slower than before. Any other details you can provide would > probably be helpful. If you’re aware if any specific file types are > causing the issue, or if all files appear to scanning slower that will also > help. > > -Micah > > > Micah Snyder > ClamAV Development > Talos > Cisco Systems, Inc. > > > > *From: *clamav-users <clamav-users-boun...@lists.clamav.net> on behalf of > Maarten Broekman via clamav-users <clamav-users@lists.clamav.net> > *Reply-To: *ClamAV users ML <clamav-users@lists.clamav.net> > *Date: *Monday, March 18, 2019 at 10:37 AM > *To: *ClamAV users ML <clamav-users@lists.clamav.net> > *Cc: *Maarten Broekman <maarten.broek...@gmail.com> > *Subject: *Re: [clamav-users] Scan very slow > > We've noticed a marked increase in scan times over the last couple of > weeks as well. From the look of it, there's something in the daily file > that's causing it. Whether this is similar to the safebrowsing issue (where > the ordering of entries in the file caused a 3000% increase in time) is > unclear. > > --Maarten Broekman > > Full scans without the daily cvd/cld: Scan time ~60seconds > Full scans with the daily from March 11th: Scan time: 84seconds > Full scans with the daily from March 17th: Scan time: 109seconds > > ~/clamav# ls -larth /tmp/clamdtest*/daily.cld > -rw-r--r-- 1 clamav clamav 110M Mar 11 04:15 /tmp/clamdtest2/daily.cld > -rw-r--r-- 1 clamav clamav 113M Mar 17 04:15 /tmp/clamdtest/daily.cld > > ~/clamav# wc /tmp/clamdtest*/daily.cld > 1514589 1517471 115031552 /tmp/clamdtest2/daily.cld > 1524782 1527664 118202368 /tmp/clamdtest/daily.cld > > Single file scans with JUST the daily.cld: > ~/clamav# time /opt/clamav/clamav/bin/clamscan -d > /tmp/clamdtest2/daily.cld test42.js > test42.js: OK > > ----------- SCAN SUMMARY ----------- > Known viruses: 1504423 > Engine version: 0.100.2 > Scanned directories: 1 > Scanned files: 1 > Infected files: 0 > Data scanned: 0.00 MB > Data read: 0.00 MB (ratio 0.00:1) > Time: 5.255 sec (0 m 5 s) > > real 0m5.260s > user 0m5.044s > sys 0m0.192s > ~/clamav# time /opt/clamav/clamav/bin/clamscan -d /tmp/clamdtest/daily.cld > test42.js > test42.js: OK > > ----------- SCAN SUMMARY ----------- > Known viruses: 1514543 > Engine version: 0.100.2 > Scanned directories: 1 > Scanned files: 1 > Infected files: 0 > Data scanned: 0.00 MB > Data read: 0.00 MB (ratio 0.00:1) > Time: 9.300 sec (0 m 9 s) > > real 0m9.329s > user 0m9.100s > sys 0m0.204s > > > > > > > On Mon, Mar 18, 2019 at 10:02 AM Yasuhiro KIMURA <y...@utahime.org> wrote: > > From: Jean-Michel via clamav-users <clamav-users@lists.clamav.net> > Subject: Re: [clamav-users] Scan very slow > Date: Mon, 18 Mar 2019 12:30:49 +0100 > > > Isn't it the second scan result ? The second analyse on same file is > faster. > > Could tou try to restart clamav-daemon and re-do the analyse with > clamdscan. > > I've tried it on 3 computers, all are above 40seconds > > It was first trial. But after restarting clamav-daemon result changed > as following. > > yasu@kusanagi[1716]% clamdscan esploso_A3TH.pdf > /home/yasu/tmp/esploso_A3TH.pdf: OK > > ----------- SCAN SUMMARY ----------- > Infected files: 0 > Time: 60.551 sec (1 m 0 s) > yasu@kusanagi[1717]% > > --- > Yasuhiro KIMURA > > _______________________________________________ > > clamav-users mailing list > clamav-users@lists.clamav.net > https://lists.clamav.net/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml > > > _______________________________________________ > > clamav-users mailing list > clamav-users@lists.clamav.net > https://lists.clamav.net/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml > > > > _______________________________________________ > > clamav-users mailing list > clamav-users@lists.clamav.net > https://lists.clamav.net/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml > > > > _______________________________________________ > > clamav-users mailing list > clamav-users@lists.clamav.net > https://lists.clamav.net/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml >
_______________________________________________ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
