2013-09-30 17:58 keltezéssel, David Raynor írta:
On Sun, Sep 29, 2013 at 5:01 AM, Boszormenyi Zoltan <[email protected]> wrote:
2013-09-29 10:26 keltezéssel, Boszormenyi Zoltan írta:
2013-09-29 04:26 keltezéssel, Benny Pedersen írta:
Is is possible to make ClamAV use less memory perhaps by repetitive
scanning with a smaller subset of the virus signature file at a time?
freshclam and clamd can use diff database dir, so if you really like to
not use main.cvd then setup freshclam.conf with database dir of freshclam,
then use rsync for daily.* from the freshclam to clamd database dir
Thanks. How can I do that? clamdoc.pdf from 0.97.8
(present on my Fedora desktop) doesn't answer that for me.
Also, there is a little problem: daily cdiff files older than
daily-17823.cdiff
are not available.
sigtool cannot seem to have an out-of-box feature to break main.cvd into
smaller pieces.
"sigtool --unpack-current=main" extracted quite a few files. main.mdb is
about 132MB,
no wonder it causes OOM.
Is there a description of the format of the files embedded in main.cvd and
daily.cvd somewhere?
I can write a utility then to break them up into arbitrary sized files
which can be treated as "3rd party" cvd files.
remember no one can force anyone to not run lowmemed :)
Indeed. :-)
imho this question is not answered here, but lets see if there is not
some one else with other solutions
just remember also there is important files in main you must have in
clamd this can be extracted with sigtool --unpack-current=main and then
moved into clamd database dir
not perfect but it works
what virus do you like to catch ?
I hope none. :-)
______________________________**_________________
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/**clamav-faq<https://github.com/vrtadmin/clamav-faq>
http://www.clamav.net/support/**ml <http://www.clamav.net/support/ml>
______________________________**_________________
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/**clamav-faq<https://github.com/vrtadmin/clamav-faq>
http://www.clamav.net/support/**ml <http://www.clamav.net/support/ml>
______________________________**_________________
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/**clamav-faq<https://github.com/vrtadmin/clamav-faq>
http://www.clamav.net/support/**ml <http://www.clamav.net/support/ml>
Zoltán,
Your idea of breaking the signature set into chunks to do repeated scans is
a workable idea. It would require a few moving parts outside of ClamAV. I
cannot write the wrapper for you but can give you some tips.
- sigtool is your best bet for unpacking CVD files. You can break the
signature files up into chunks after that.
- Some signature files will need to be loaded for all chunks. They load
configuration values or prevent FP.
What does "prevent FP" mean?
Based on current main.cvd and
daily.cvd, the extensions of these files are: [ .ftm .cfg .fp .ign2 ] This
list may change in the future, but works for now.
What about the other files in daily.cvd?
.db .ftm .hdb .hdu .idb .ign (without the 2)
.info .ldb .ldu .mdu .ndb .ndu .pdb .wdb .zmd
Also, what about the "missing" files from main.cvd?
.ftm .cfg .ign2 are not present but there are .db .fp .hdb .ndb and .zmd
- If you use bytecode.cvd, that should also be in all chunks.
If I understand you correctly, the final broken-up database should
consist of several directories and bytecode.cvd present in every one of them?
Or the contents of bytecode.cvd (.cbc files) should be packed into each cvd?
- Many of the existing signature types are 1 signature per line.
I have discovered it already, the text file can be nicely broken up into pieces.
You can
fragment those files as you see fit without breaking signatures as long as
you keep whole lines. The mdb signature files are the largest and will
probably need to be divided into the most pieces. The memory use is not
purely linear with number of signatures. You will have to experiment to
find out what works for you.
At first thought, 10MB chunks created from the unpacked .mdb would do nicely.
- You will need to collate results. Files that alert on only 1 signature
_will_ report clean from scans that do not include that signature! BE
CAREFUL.
Of course.
Disclaimer (because I have to): YMMV. I cannot guarantee your results or
support your configuration.
Of course.
That said, your idea should let you find a way to operate in your
environment and keep scanning.
Good luck,
Dave R.
Thank you very much for the detailed explanation.
With "sigtool --unpack-current=main" I already found out the .mdb file
needs to be broken up. My question is whether there is any cross-dependency
between different files and signatures? If there is, can it be detected somehow?
Best regards,
Zoltán Böszörményi
_______________________________________________
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/support/ml
