Re: [Clamav-devel] ClamAV Algorithms
Hi Edwin,
Thanks for your help. I used --dev-ac-only on the command line and that
does the trick. I couldn't find anything about using --dev-ac-only in the
manual page.
Regards,
Jerry
Date: Thu, 14 Jul 2011 15:03:12 +0300
From: edwinto...@gmail.com
To: clamav-devel@lists.clamav.net
Subject: Re: [Clamav-devel] ClamAV Algorithms
On 07/12/2011 02:11 AM, Jerry 270 wrote:
Hi Edwin,
Thanks for your reply. I am doing a Masters degree for which the
research is analyzing investigating malware. I am interested in
evaluating algorithms used in anti-virus software, but just investigating
whether this is a possibility at the moment. The research projects goal is
to define a problem domain, a scenario in which the problem to be
investigated exists. Within this problem domain, a research question is
posed. This is the question that the project will seek to answer.
I enabled DevAVOnly and only the AC signatures appear to be loaded
when the config file is reread but when I do a scan of some files the debug
information appears to suggest that BM signatures are loaded for GENERIC
and PE.
If you are using clamscan then use --dev-ac-only. I get 0 BM sigs:
LibClamAV debug: Matcher[0]: GENERIC: AC sigs: 35862 (reloff: 21, absoff: 0)
BM sigs: 0 (reloff: 0, absoff: 0) maxpatlen 470 (ac_only mode)
LibClamAV debug: Using filter for trie 1
LibClamAV debug: Matcher[1]: PE: AC sigs: 59482 (reloff: 47699, absoff: 0) BM
sigs: 0 (reloff: 0, absoff: 0) maxpatlen 468 (ac_only mode)
LibClamAV debug: Matcher[2]: OLE2: AC sigs: 1726 (reloff: 0, absoff: 0) BM
sigs: 0 (reloff: 0, absoff: 0) maxpatlen 176 (ac_only mode)
LibClamAV debug: Matcher[3]: HTML: AC sigs: 5773 (reloff: 0, absoff: 0) BM
sigs: 0 (reloff: 0, absoff: 0) maxpatlen 799 (ac_only mode)
LibClamAV debug: Using filter for trie 4
LibClamAV debug: Matcher[4]: MAIL: AC sigs: 1146 (reloff: 0, absoff: 0) BM
sigs: 0 (reloff: 0, absoff: 0) maxpatlen 255 (ac_only mode)
LibClamAV debug: Matcher[5]: GRAPHICS: AC sigs: 23 (reloff: 0, absoff: 0) BM
sigs: 0 (reloff: 0, absoff: 0) maxpatlen 227 (ac_only mode)
LibClamAV debug: Matcher[6]: ELF: AC sigs: 47 (reloff: 29, absoff: 0) BM
sigs: 0 (reloff: 0, absoff: 0) maxpatlen 400 (ac_only mode)
LibClamAV debug: Using filter for trie 7
LibClamAV debug: Matcher[7]: ASCII: AC sigs: 1568 (reloff: 0, absoff: 0) BM
sigs: 0 (reloff: 0, absoff: 0) maxpatlen 492 (ac_only mode)
LibClamAV debug: Matcher[8]: NOT USED: AC sigs: 0 (reloff: 0, absoff: 0) BM
sigs: 0 (reloff: 0, absoff: 0) maxpatlen 0 (ac_only mode)
LibClamAV debug: Matcher[9]: MACH-O: AC sigs: 0 (reloff: 0, absoff: 0) BM
sigs: 0 (reloff: 0, absoff: 0) maxpatlen 0 (ac_only mode)
What should DevACDepth be set to?
If AC is used for signatures containing wildcards and BM is used for
signatures without wildcards is it possible to scan using just one type of
signature and test the performance of each algorithm that way?
Well you won't be able to load the AC signatures into BM (BM doesn't support
the wildcards), so as a first step you would probably be to remove
the signatures that require AC from the DB.
You can use 'sigtool --unpack-current main' and 'sigtool --unpack-current
daily' to unpack the databases.
And then load the DB as by default (into BM), and with --dev-ac-only (into
AC), and compare them that way.
Also note that the BM algo has an optimization when signatures are tied to a
specific offset (PE for example).
How is prefiltering disabled?
Comment out this 'if' in matcher-ac.c:
if (cli_mtargets[root-type].enable_prefiltering dconf_prefiltering) {
Best regards,
--Edwin
___
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net
___
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net
Re: [Clamav-devel] ClamAV Algorithms
On 07/12/2011 02:11 AM, Jerry 270 wrote:
Hi Edwin,
Thanks for your reply. I am doing a Masters degree for which the
research is analyzing investigating malware. I am interested in evaluating
algorithms used in anti-virus software, but just investigating whether this
is a possibility at the moment. The research projects goal is to define a
problem domain, a scenario in which the problem to be investigated exists.
Within this problem domain, a research question is posed. This is the
question that the project will seek to answer.
I enabled DevAVOnly and only the AC signatures appear to be loaded when
the config file is reread but when I do a scan of some files the debug
information appears to suggest that BM signatures are loaded for GENERIC and
PE.
If you are using clamscan then use --dev-ac-only. I get 0 BM sigs:
LibClamAV debug: Matcher[0]: GENERIC: AC sigs: 35862 (reloff: 21, absoff: 0) BM
sigs: 0 (reloff: 0, absoff: 0) maxpatlen 470 (ac_only mode)
LibClamAV debug: Using filter for trie 1
LibClamAV debug: Matcher[1]: PE: AC sigs: 59482 (reloff: 47699, absoff: 0) BM
sigs: 0 (reloff: 0, absoff: 0) maxpatlen 468 (ac_only mode)
LibClamAV debug: Matcher[2]: OLE2: AC sigs: 1726 (reloff: 0, absoff: 0) BM
sigs: 0 (reloff: 0, absoff: 0) maxpatlen 176 (ac_only mode)
LibClamAV debug: Matcher[3]: HTML: AC sigs: 5773 (reloff: 0, absoff: 0) BM
sigs: 0 (reloff: 0, absoff: 0) maxpatlen 799 (ac_only mode)
LibClamAV debug: Using filter for trie 4
LibClamAV debug: Matcher[4]: MAIL: AC sigs: 1146 (reloff: 0, absoff: 0) BM
sigs: 0 (reloff: 0, absoff: 0) maxpatlen 255 (ac_only mode)
LibClamAV debug: Matcher[5]: GRAPHICS: AC sigs: 23 (reloff: 0, absoff: 0) BM
sigs: 0 (reloff: 0, absoff: 0) maxpatlen 227 (ac_only mode)
LibClamAV debug: Matcher[6]: ELF: AC sigs: 47 (reloff: 29, absoff: 0) BM sigs:
0 (reloff: 0, absoff: 0) maxpatlen 400 (ac_only mode)
LibClamAV debug: Using filter for trie 7
LibClamAV debug: Matcher[7]: ASCII: AC sigs: 1568 (reloff: 0, absoff: 0) BM
sigs: 0 (reloff: 0, absoff: 0) maxpatlen 492 (ac_only mode)
LibClamAV debug: Matcher[8]: NOT USED: AC sigs: 0 (reloff: 0, absoff: 0) BM
sigs: 0 (reloff: 0, absoff: 0) maxpatlen 0 (ac_only mode)
LibClamAV debug: Matcher[9]: MACH-O: AC sigs: 0 (reloff: 0, absoff: 0) BM sigs:
0 (reloff: 0, absoff: 0) maxpatlen 0 (ac_only mode)
What should DevACDepth be set to?
If AC is used for signatures containing wildcards and BM is used for
signatures without wildcards is it possible to scan using just one type of
signature and test the performance of each algorithm that way?
Well you won't be able to load the AC signatures into BM (BM doesn't support
the wildcards), so as a first step you would probably be to remove
the signatures that require AC from the DB.
You can use 'sigtool --unpack-current main' and 'sigtool --unpack-current
daily' to unpack the databases.
And then load the DB as by default (into BM), and with --dev-ac-only (into AC),
and compare them that way.
Also note that the BM algo has an optimization when signatures are tied to a
specific offset (PE for example).
How is prefiltering disabled?
Comment out this 'if' in matcher-ac.c:
if (cli_mtargets[root-type].enable_prefiltering dconf_prefiltering) {
Best regards,
--Edwin
___
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net
Re: [Clamav-devel] ClamAV Algorithms
On 2011-07-11 01:00, Jerry 270 wrote: Hi, I am a newbie to ClamAV so require help please. I am doing a research project and would like to evaluate the BM and AC algorithms used by ClamAV. Is there anyway to get ClamAV to use either BM or AC on their own so scanning speed tests can be conducted for each algorithm separately? I have read on this list that you can't get BM to run on its own. What is the best way to compare the two algorithms and can someone give me more information on how ClamAV uses or chooses between BM and AC please? There are some AC/BM discussion in the archive, see for example: http://lurker.clamav.net/message/20100426.103047.eb6fd9d0.en.html http://lurker.clamav.net/message/20100427.131931.b705e603.en.html http://lurker.clamav.net/message/20081204.212941.c9fa45c2.en.html You can use DevACOnly to use only the AC algorithm for everything (there is no equivalent for BM), Other than that you can use tools such as 'oprofile', or 'perf record / perf report' to see how much time is spent in functions from matcher-ac.c and how much time in those from matcher-bm.c. Also note that there is the prefiltering step too, if you want to measure just the AC/BM performance, you should disable that (although you'll loose performance by doing so). What are your research project's goals? Best regards, --Edwin ___ http://lurker.clamav.net/list/clamav-devel.html Please submit your patches to our Bugzilla: http://bugs.clamav.net
