Taizo, The reason for the difference is that there are a lot of subsignatures used in the published set of logical signatures that begin with some number of zeroes (more so than the majority of random byte sequences), so the ClamAV pattern matcher has to do a lot more work on the all-zeroes file determining that none of the signatures fully match. Also, there are likely some short all-zero subsigs that get used based on how certain subsignature features are implemented, and these can also affect performance on large files consisting mainly of zeroes.
Hope that helps! Thanks for asking about this - your observation is a good reminder to us that a large all-zero file makes a good test case for catching signatures that might have egregious performance impacts. :) -Andrew Andrew Williams Malware Research Team Cisco Talos On Tue, Jul 9, 2019 at 11:07 PM Taizo ITO <[email protected]> wrote: > Hello, > > I'm trying to get some stats on how long a scan takes by different > size, but I encountered an unexpected behavior when scanning a file > generated in a specific way. > A scan for a dummy file filled with /dev/zero takes much longer than > with /dev/urandom. I think the processing time should be the same or > less. > I'd like to know how to avoid this problem because that may cause stuck > service. > I'm using ClamAV version 0.101.2/25504. > > You can reproduce the problem by doing the following. > > A 10MB file with /dev/zero: Takes almost 7 times as long as /dev/urandom. > ``` > $ dd if=/dev/urandom of=dummy-dd-10MB-with-urandom.iso bs=10MB count=1 > $ dd if=/dev/zero of=dummy-dd-10MB-with-zero.iso bs=10MB count=1 > $ ls -ltr dummy-dd-10MB-with-* > -rw-r--r-- 1 user user 10000000 Jul 9 03:41 dummy-dd-10MB-with-zero.iso > -rw-r--r-- 1 user user 10000000 Jul 9 03:41 dummy-dd-10MB-with-urandom.iso > > $ time (echo "SCAN dummy-dd-10MB-with-zero.iso" | nc -U > /var/run/clamd.scan/clamd.sock) > dummy-dd-10MB-with-zero.iso: OK > real 0m4.056s > user 0m0.008s > sys 0m0.004s > > $ time (echo "SCAN dummy-dd-10MB-with-urandom.iso" | nc -U > /var/run/clamd.scan/clamd.sock) > dummy-dd-10MB-with-urandom.iso: OK > real 0m0.569s > user 0m0.012s > sys 0m0.000s > ``` > > > A 250MB file with /dev/zero: Takes almost 8 times as long as /dev/urandom. > ``` > $ dd if=/dev/zero of=dummy-dd-250MB-with-zero.iso bs=25MB count=10 > $ dd if=/dev/urandom of=dummy-dd-250MB-with-urandom.iso bs=25MB count=10 > $ ls -ltr dummy-dd-250MB-with-* > -rw-r--r-- 1 user user 250000000 Jul 9 03:44 > dummy-dd-250MB-with-urandom.iso > -rw-r--r-- 1 user user 250000000 Jul 9 03:44 dummy-dd-250MB-with-zero.iso > > $ time (echo "SCAN dummy-dd-250MB-with-zero.iso" | nc -U > /var/run/clamd.scan/clamd.sock) > dummy-dd-250MB-with-zero.iso: OK > real 1m42.949s > user 0m0.009s > sys 0m0.003s > > $time (echo "SCAN dummy-dd-250MB-with-urandom.iso" | nc -U > /var/run/clamd.scan/clamd.sock) > dummy-dd-250MB-with-urandom.iso: OK > real 0m12.905s > user 0m0.004s > sys 0m0.007s > ``` > > Thanks. > > -- > Taizo Ito > > _______________________________________________ > > clamav-users mailing list > [email protected] > https://lists.clamav.net/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml >
_______________________________________________ clamav-users mailing list [email protected] https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
