On Wednesday 05 April 2006 18:25, Daniel T. Staal wrote: > On Wed, April 5, 2006 1:08 pm, Rob MacGregor said: > > WTF are you doing accepting email's at 200 MB? There are far more > > appropriate methods of file transfer than SMTP! > > But they all require more complicated and lasting setups than SMTP, for > a specific set of senders/receivers. > > If you want to send a large file between two people who are likely to > never send each other a file again, SMTP is a quick and easy way to do > it. It is not the most efficient, but efficiency is not a primary > concern in this case. Having ClamAV not fall over would be nice. ;)
Thanks for your comments, I'm glad someone can see my point of view here :) If I zip up five 35Mb files into a zip archive (resulting size 110Mb), Clam scans it in 30 seconds and needs less than 20 Mb of VmData including all the patterns etc. Clamscan reports "Data scanned: 108.61 MB" which is the same size as the unzipped contents. If I enclose those identical files into an email, which is just another form of encapsulation, then Clam will take over 3 minutes and requires enough VmData to keep that entire email in memory, plus apparently at least one copy of the attachment it is working on - total 300Mb required to scan the exact same files. Clamscan also reports "Data scanned: 662.11 MB" - six times as much as was actually in the email, over three times as much as the Base64 contents. So Clam scanning an mbox format MIME archive is six times slower and over 10 times greedier for RAM than scanning the same files in a zip archive. I haven't finished tracking through the code yet to find why it's coded like this and hence where this inefficiency comes from. I accept it was probably an efficient way to handle small mails. But I can already see that mbox.c does a lot of loading things into RAM, when it could probably use the disk copy and just keep headers and pointers. BTW - to the person who asked how other AV's handle this - I will get some figures and report back. Nick _______________________________________________ http://lurker.clamav.net/list/clamav-users.html
