On 24 Apr 2012 at 18:11, Steve Basford wrote: > > Has anyone else seen these kinds of delays? Is there any way to get > > these databases to load faster or to allow ClamAV to continue scanning > > when the database is being reloaded? > > Sorry for the briefness here, as I'm currently sorting out my home > internet access... > > For those having issues: > > a) what databases are loaded > b) what OS are you running > > It could be, as someone else suggested a tipping point in memory, but > we need to get a handle on db's used etc. > > Perhaps we can then get a set of test data and create a bugzilla clamav > entry....
I don't know if this can help speeding up the process but I collected some
statistics on
clamscan of a small file (wallclock duration: ~25sec):
# ./DTraceToolkit-0.96/procsystime -cen clamscan
Elapsed Times for processes clamscan,
SYSCALL TIME (ns)
getuid 3750
gtime 4083
lwp_sigmask 4667
sigpending 4667
systeminfo 5750
times 6167
getpid 7417
sysconfig 11332
fstat 15082
setcontext 16250
getrlimit 17751
fcntl 26000
dup 28750
stat 36251
fsat 44833
mprotect 46000
pread 51416
lstat 53250
schedctl 57500
getdents64 107667
access 154583
ioctl 209083
write 301667
lseek 336754
fstat64 435166
resolvepath 629499
memcntl 749498
llseek 816746
open 1308664
stat64 1714168
brk 1799754
close 1835584
mmap 24644318
munmap 125520181
read 1469157031
Syscall Counts for processes clamscan,
SYSCALL COUNT
fsat 1
getuid 1
gtime 1
lstat 1
lwp_sigmask 1
pread 1
rexit 1
schedctl 1
sigpending 1
systeminfo 1
times 1
fstat 2
getpid 2
getrlimit 2
mprotect 2
setcontext 2
stat 2
sysconfig 3
getdents64 4
dup 6
fcntl 7
access 9
write 10
memcntl 20
ioctl 29
resolvepath 29
open 46
close 52
fstat64 58
lseek 88
stat64 101
brk 170
llseek 175
munmap 612
mmap 674
read 13573
At first glance it looks like the 13573 reads (of 8KB, as seen with another
command) take
most of the system time (1.5 sec). Larger reads might help, but it seems that
it is in userland
that most of the time is spent. Inside the process, after executing about 19000
the __udivdi3
function, there is a long delay (most of the execution time), and then about
69000 times the
__ashidi3 function. Now, looking at all functions occurences (from binary or
libraries), these
are the most often called ones:
FUNCTION COUNT
...
cli_htu32_next 201030
cli_isnumber 202744
atoi 202763
__mul64 206909
cli_caloff 226946
fgets 242252
memccpy 244999
cli_strtok 252988
_realbufend 255751
getxfdat 255774
sprintf 259956
_ndoprnt 259975
ferror 259975
cli_mpool_hex2str 262241
cli_malloc 314546
qcompare 355340
malloc 370694
_malloc_unlocked 372151
free 495212
_free_unlocked 496305
mpool_calloc 558708
strcmp 568989
memset 589919
mutex_lock 869471
mutex_lock_impl 869471
mutex_unlock 869471
cli_mpool_strdup 1211577
hm_addhash 1243854
cli_htu32_find 1243855
strtol 1243908
cli_strtokenize 1407183
strncpy 1413460
cli_chkign 1437106
cli_bm_scanbuff 1437108
cli_chomp 1438745
cli_dbgets 1439239
cli_mpool_virname 1470902
cli_hex2str_to 1506095
strcpy 1525859
hm_sort 1595114
memcmp 1664141
mpool_realloc2 2541350
allocate_aligned 2591715
mpool_realloc 2758207
mpool_malloc 2972354
mpool_free 2989609
to_bits 5564069
strlen 7525133
from_bits 9835704
strchr 13787666
memcpy 25818611
More than 25 000 000 memcpy for 1446134 sigs. It is not due to the perl script
I'm scanning
as it is only 2450 bytes long.
HTH
Pierre
_______________________________________________
Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
http://www.clamav.net/support/ml
