On Fri, May 12, 2006 at 05:32:20PM +0300, Denis Vlasenko wrote: > On Thursday 11 May 2006 15:31, [EMAIL PROTECTED] wrote: > > Hi, > > > > I'm currently working on a port of clamav for our embedded firewall > > appliance "mGuard". Right now I am trying to implement the database > > loading in a way that does not require extra space in the tmp directory > > (an extra 10 MB of RAMdisk space to load the database is a problem > > on a device with only 64 MB of RAM). So everything is parsed from > > the 512 byte blocks read by gzread(), needing only one extra linebuffer > > of < 1kB and no unpacked database files in tmp space. > > Nice! > > I always wondered why clamav creates those temp files. > That slows down things quite a bit. A bad design. > I really like that someone want to improve it. > > In the longer term: > > I think virus database should be stored in such a format > that it can be just mmap'ed and directly used, without > reading it with read() syscalls. This will eliminate > yet another mem-to-mem copying. > > Something like cdb databases of Dan Bernstein: > http://cr.yp.to/cdb.html
IMHO reading the database is not a speed critical task, it just happens once a while after an update. I dont know, is the scanengine startup time really relevant in many scenarios ? It is however in some cases very critical how much space the database consumes in terms of internet transfer, in the filesystem and in RAM (think of an embedded device using a flash filesystem and getting its database updates via an expensive GSM uplink). So a tradeof of startup time against database size could be very beneficial IMO. In an embedded environment it would be worth to consider better methods than gzip for database compression , like bzip2, 7-zip (http://www.7-zip.org/sdk.html), AdvanceCOMP (http://advancemame.sourceforge.net/comp-readme.html), etc. Another idea would be to dump the virus names in favour of some binary index number, which can be used to lookup the virus name later in case of an infection. Cheers Peter -- Peter Vollmer Software Engineer Innominate Security Technologies AG /device attached security/ Albert-Einstein-Str. 14 D-12489 Berlin, Germany www.innominate.com _______________________________________________ http://lurker.clamav.net/list/clamav-devel.html
