Hi there, On Sun, 8 Sep 2019, Markus Kolb wrote:
I'd like to get some thoughts of reducing the memory footprint of clamav (clamd)...
My thought: Once upon a time they used to say that hardware accounted for 90% of the cost of a computer system; then they used to say it was more like 50/50; more recently, by comparison with the software costs, hardware costs approximately nothing. Short version: memory's cheap.
Do you have already some ideas of the way to introduce a feature like this?
Speaking personally, no.
Is my assumption correct that the loaded signature database in memory is the biggest part?
Don't assume things. Measure them.
So my idea would be to do introduce some kind of optional/configurable round-robin loading of the signatures in limited blocks. Scanning with the loaded signatures block, loading the next signatures block from db files and scanning, and so on, until all signatures are used.
Perhaps you haven't been reading recent posts on the users' list.
With the next file to scan same procedure again... Of course this would mean to slow down the scan process, but would free memory on the system.
The *scanning* speed isn't going to be the problem with your idea.
... I might have missed something to think about.
The scanning engine was designed to scan for millions of signatures very efficiently. But it loads them rather slowly. Even if you were to split the database into parts which were one tenth the size of the existing databases, they would each take seconds to load. But a scan typically takes a fraction of a second. So I think what you've missed is the speed of *loading* the signatures. Unless you have some way of improving that (and everyone here would be *very* pleased to see that) then your idea will have little to recommend it to most ClamAV users. There might be a way to run multiple daemons, each with part of the database already loaded, which could be paged in/out of swap quicker than it would be to dump and reload the separate database parts. The idea is pure speculation on my part, and I wouldn't think it worth pursuing unless I wanted to run ClamAV on a Raspberry Pi on the ISS. What's your use case? -- 73, Ged. _______________________________________________ clamav-devel mailing list clamav-devel@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-devel Please submit your patches to our Bugzilla: http://bugzilla.clamav.net Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
