Re: [clamav-users] Heuristics.Limits.Exceeded FOUND
Much of that time is almost certainly being consumed by loading the signature database into RAM. How long does it take using clamdscan? Sent from my iPad -Al- On Apr 6, 2020, at 12:29, Paul Kosinski via clamav-users wrote: > > It *does* take more than 120 secs for the clamscan command to fully > scan the 62 MB Firefox installation file (.tar.bz2). Trying the scan > with the default clamscan limits results in 62 MB "Data read" but > *zero* "Data scanned"! smime.p7s Description: S/MIME cryptographic signature ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] Heuristics.Limits.Exceeded FOUND
Micah, It *does* take more than 120 secs for the clamscan command to fully scan the 62 MB Firefox installation file (.tar.bz2). Trying the scan with the default clamscan limits results in 62 MB "Data read" but *zero* "Data scanned"! Since I previously had run afoul of file size limits, I had written a wrapper script that set all the "--max-*" limits to values that should not cause any unnecessary failures. The problem I ran into with 0.102.x was that the "--help" info for the clamscan command's "--max-scantime" was incomplete. I had set the "--max-scantime" limit to 999, assuming it was seconds. It never occurred to me that it would be milliseconds, especially since the clamscan command can't even load the DB in under a second. (Milliseconds would be reasonable for clamd usage, I suppose.) When somebody pointed out that the max scan time was really in msecs, I updated my wrapper script and everything worked nicely, like 0.101.x. Now, scanning the big Firefox installation file takes well over 120 secs real time, to wit (expanding the wrapper): time clamscan --alert-exceeds-max=yes --max-scantime=99 --max-scansize=4090M --max-filesize=4090M --max-files=3 --max-recursion=30 --pcre-match-limit=9 --pcre-max-filesize=9 firefox-68.6.1-esr-64.tar.bz2 firefox-68.6.1-esr-64.tar.bz2: OK --- SCAN SUMMARY --- Known viruses: 6797620 Engine version: 0.102.1 Scanned directories: 0 Scanned files: 1 Infected files: 0 Data scanned: 622.26 MB Data read: 62.06 MB (ratio 10.03:1) Time: 140.191 sec (2 m 20 s) real2m20.219s user2m17.212s sys 0m2.820s Paul P.S. This is on an "Intel(R) Core(TM) i7-3820 CPU @ 3.60GHz" with 32 GB RAM. On Mon, 6 Apr 2020 15:23:42 + "Micah Snyder (micasnyd)" wrote: > Paul, > > Are you seeing many files that take longer than 2 minutes to scan? > We thought the default scan time limit was already quite high at 2 > minutes. > > -Micah > > On 4/4/20, 1:47 AM, "clamav-users on behalf of Paul Kosinski via > clamav-users" clamav-users@lists.clamav.net> wrote: > > "If one is overriding a default value by providing it on the > command line, you should know what you're doing. Guessing is never a > good idea, especially if (like here) the documentation is lacking." > > "It was noted in the list of notable changes in 0.102.0 ... which > Paul *must* have read, otherwise he would *not* have known of the > existence of this parameter". Really? > > Does issuing "clamscan --help", and reading its output of 700 > words on 103 lines (according to wc), including one line about > "--max-scantime", constitute guessing? Who knew? > > P.S. Up until 0.102.0, direct use of the clamscan command worked > well for files like the Firefox download. Starting with 0.102.0, > clamscan started giving Heuristic Limit errors. Since there was no > indication as to *which* Limit was hit, I read the "--help" to see > what to do. ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] Heuristics.Limits.Exceeded FOUND
Paul, Are you seeing many files that take longer than 2 minutes to scan? We thought the default scan time limit was already quite high at 2 minutes. -Micah On 4/4/20, 1:47 AM, "clamav-users on behalf of Paul Kosinski via clamav-users" wrote: "If one is overriding a default value by providing it on the command line, you should know what you're doing. Guessing is never a good idea, especially if (like here) the documentation is lacking." "It was noted in the list of notable changes in 0.102.0 ... which Paul *must* have read, otherwise he would *not* have known of the existence of this parameter". Really? Does issuing "clamscan --help", and reading its output of 700 words on 103 lines (according to wc), including one line about "--max-scantime", constitute guessing? Who knew? P.S. Up until 0.102.0, direct use of the clamscan command worked well for files like the Firefox download. Starting with 0.102.0, clamscan started giving Heuristic Limit errors. Since there was no indication as to *which* Limit was hit, I read the "--help" to see what to do. On Fri, 03 Apr 2020 23:30:57 +0200 Arjen de Korte via clamav-users wrote: > Citeren Kris Deugau : > > > Arjen de Korte via clamav-users wrote: > >> Citeren Paul Kosinski via clamav-users > >> : > > > >>> However, applying clamscan to this file (which was slightly > >>> renamed by my download script to be more readable) results in the > >>> following output: > >>> > >>> clamscan --alert-exceeds-max=yes --max-scantime=999 > >>> --max-scansize=4090M --max-filesize=4090M --max-files=3 > >>> --max-recursion=30 --pcre-match-limit=9 > >>> --pcre-max-filesize=9firefox-68.6.1-esr-64.tar.bz2 > >>> > > > >> Before writing this whole rant, you have not considered checking > >> which of the options might have triggered this? You've reduced > >> the --max-scantime from the default 120 seconds to under 1 second > >> and still wonder why this breaks? Really? > > > > That option seems to be missing from the man page entirely: > > > > $ dpkg -l clamav > > ii clamav 0.102.1+dfsg-0+deb10u2 amd64 [...] > > $ zgrep scantime /usr/share/man/man1/clamscan.1.gz > > $ > > > > > > and does not specify units in the --help text: > > > > $ clamscan --help > > [...] > > --max-scantime=#nScan time longer than > > this will be skipped and assumed clean > > [...] > > > > Absent any documentation, I would reasonably assume this to be in > > seconds, not milliseconds. > > > > I have no idea if you're wrong about this being the cause, but > > without diving into the source, Paul's use of that option looks > > entirely reasonable to me. > > If one is overriding a default value by providing it on the > commandline, you should know what you're doing. Guessing is never a > good idea, especially if (like here) the documentation is lacking. > It was noted in the list of notable changes in 0.102.0 (see > https://blog.clamav.net/2019/10/clamav-01020-has-been-released.html) > which Paul must have read, otherwise he would not have known of the > existence of this parameter. > > > -kgd ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] Squid + ClamAV
On 06/04/2020 15:53, Andrea Venturoli via clamav-users wrote: On 2020-04-02 08:14, Andrea Venturoli wrote: P.S. I'm investigatint your other message about the reload patch. Patch is working. However almost nothing has changed: from the logs I see DB reloads twice/three times per day... hard to hit if you try :) and in the meanwhile I still see slowness (which comes from something else, then). From my experience sometimes database check and reload is triggered when a scan is initiated. I started noticing it when I reverted back from the threaded reload patch. Good luck Reio ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] Squid + ClamAV
On 2020-04-02 08:14, Andrea Venturoli wrote: P.S. I'm investigatint your other message about the reload patch. Patch is working. However almost nothing has changed: from the logs I see DB reloads twice/three times per day... hard to hit if you try :) and in the meanwhile I still see slowness (which comes from something else, then). bye & Thanks av. ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
