Re: [clamav-users] freshclam checks database every time
Am 10.04.2013 15:05 schrieb Andreas Schulze: symptom: freshclam needs 3..4 seconds to finish also in the case where *no* updates are available. No comments on this topic anymore... Thats worse because freshclam still steal cputime here :-( To dive into the problem I describe the problem again: keep an eye on the timestamps! # echo 'DatabaseMirror clamav.datev.de' /tmp/freshclam.conf # (/usr/bin/freshclam --config-file=/tmp/freshclam.conf --verbose --debug --no-dns; echo $?) 21 | tai64n | tai64nlocal 2013-04-10 14:49:25.241597500 Current working dir is /var/lib/clamav - starting freshclam ... 2013-04-10 14:49:25.253432500 bytecode.cvd is up to date (version: 214, sigs: 41, f-level: 63, builder: neo) - after 12ms freshclam detects, that no updates are available 2013-04-10 14:49:25.253533500 LibClamAV debug: Initialized 0.97.7 engine 2013-04-10 14:49:25.253533500 LibClamAV debug: in cli_cvdload() - now freshclam load the entire db ... 2013-04-10 14:49:28.147515500 LibClamAV debug: Phishcheck cleaned up - and after ~3s freshclam finished PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 4940 clamav20 0 13212 3256 2588 R 100 0.0 0:01.35 freshclam - 3 seconds of 100% cpu usage for nothing! I agree if freshclam load a *new* db in case of a *new* pattern version to verify the data are valid. But if no update was available, thats total unnecessary! Is there any clamav developer who could point me into the code or explain the logic? Thanks! Any feedback is welcome:-) I looked in the source but did not found a relevant place to disable that behaviour. A patches or a pointer into the source would be nice :-) Thats still my intention ... -- Andreas Schulze Internetdienste | P252 DATEV eG 90329 Nürnberg | Telefon +49 911 319-0 | Telefax +49 911 319-3196 E-Mail info @datev.de | Internet www.datev.de Sitz: 90429 Nürnberg, Paumgartnerstr. 6-14 | Registergericht Nürnberg, GenReg Nr.70 Vorstand Prof. Dieter Kempf (Vorsitzender) Dipl.-Kfm. Wolfgang Stegmann (stellvertretender Vorsitzender) Dipl.-Kfm. Michael Leistenschneider Dipl.-Kfm. Dr. Robert Mayr Jörg Rabe v. Pappenheim Dipl.-Vw. Eckhard Schwarzer Vorsitzender des Aufsichtsrates: Reinhard Verholen ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] freshclam checks database every time
On Fri, Jun 21, 2013 at 7:28 AM, Andreas Schulze andreas.schu...@datev.dewrote: I agree if freshclam load a *new* db in case of a *new* pattern version to verify the data are valid. But if no update was available, thats total unnecessary! Is there any clamav developer who could point me into the code or explain the logic? Freshclam tests loading the databases, even if they haven't been updated, to ensure that they haven't become corrupted (hard disk errors, manual tampering, etc.). Since Freshclam is the tool that updates the databases, this is the perfect place to test their integrity. If they fail to load, Freshclam can download fresh copies. Thanks, Shawn ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] freshclam checks database every time
Am 12.04.2013 20:27 schrieb Matus UHLAR - fantomas: On 12.04.13 19:58, Andreas Schulze wrote: But back to my main problem. clamscan wastes 3 seconts time loading the complete engine every time it is called. loading virus databage to memory and parsing it is really CPU-intensive process. That's why we recommend running clamd daemon and using clamdscan. Sorry, my fault: s/clamscan/freshclam/ for clamscan you are right. But I mean the updater... -- Andreas Schulze Internetdienste | P252 DATEV eG 90329 Nürnberg | Telefon +49 911 319-0 | Telefax +49 911 319-3196 E-Mail info @datev.de | Internet www.datev.de Sitz: 90429 Nürnberg, Paumgartnerstr. 6-14 | Registergericht Nürnberg, GenReg Nr.70 Vorstand Prof. Dieter Kempf (Vorsitzender) Dipl.-Kfm. Wolfgang Stegmann (stellvertretender Vorsitzender) Dipl.-Kfm. Michael Leistenschneider Dipl.-Kfm. Dr. Robert Mayr Jörg Rabe v. Pappenheim Dipl.-Vw. Eckhard Schwarzer Vorsitzender des Aufsichtsrates: Reinhard Verholen ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] freshclam checks database every time
Am 12.04.2013 11:19 schrieb Chuck Swiger: The data available to me suggests that ClamAV has seen ~736 database updates since the beginning of the year, for an average update frequency of ~3.5 hours between updates. that fine. I could update not once a minute but every three hours. But in the worst case I use old pattern while new pattern are available since 3 hours. Thats what I like to avoid. For that reason I update so often. Frankly, trying to update more often than once an hour is just a waste of resources. Do you have better ideas to minimize the time between - new pattern is available on $my_local_update_mirror - new pattern is used on $my_local_clamd ?? Thanks Andreas -- Andreas Schulze Internetdienste | P252 DATEV eG 90329 Nürnberg | Telefon +49 911 319-0 | Telefax +49 911 319-3196 E-Mail info @datev.de | Internet www.datev.de Sitz: 90429 Nürnberg, Paumgartnerstr. 6-14 | Registergericht Nürnberg, GenReg Nr.70 Vorstand Prof. Dieter Kempf (Vorsitzender) Dipl.-Kfm. Wolfgang Stegmann (stellvertretender Vorsitzender) Dipl.-Kfm. Michael Leistenschneider Dipl.-Kfm. Dr. Robert Mayr Jörg Rabe v. Pappenheim Dipl.-Vw. Eckhard Schwarzer Vorsitzender des Aufsichtsrates: Reinhard Verholen ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] freshclam checks database every time
Am 11.04.2013 15:50, schrieb Greg Folkert: Is that checking your *OWN* mirror? If not, you are being throttled. sure, I ask my own server of course. But back to my main problem. clamscan wastes 3 seconts time loading the complete engine every time it is called. Notice the timestamps! # echo 'DatabaseMirror clamav.datev.de' /tmp/freshclam.conf # (/usr/bin/freshclam --config-file=/tmp/freshclam.conf --verbose --debug --no-dns; echo $?) 21 | tai64n | tai64nlocal 2013-04-10 14:49:25.241597500 Current working dir is /var/lib/clamav 2013-04-10 14:49:25.241655500 Max retries == 3 2013-04-10 14:49:25.241840500 ClamAV update process started at Wed Apr 10 14:49:25 2013 2013-04-10 14:49:25.241841500 Using IPv6 aware code 2013-04-10 14:49:25.241935500 If-Modified-Since: Tue, 11 Oct 2011 14:34:20 GMT 2013-04-10 14:49:25.244810500 Connected to clamav.datev.de (IP: 193.27.49.165). 2013-04-10 14:49:25.244812500 Trying to retrieve CVD header ofhttp://clamav.datev.de/main.cvd 2013-04-10 14:49:25.246660500 OK 2013-04-10 14:49:25.246665500 main.cvd is up to date (version: 54, sigs: 1044387, f-level: 60, builder: sven) 2013-04-10 14:49:25.246758500 If-Modified-Since: Wed, 10 Apr 2013 02:36:19 GMT 2013-04-10 14:49:25.246759500 Reading CVD header (daily.cvd): Connected to clamav.datev.de (IP: 193.27.49.165). 2013-04-10 14:49:25.248873500 Trying to retrieve CVD header ofhttp://clamav.datev.de/daily.cvd 2013-04-10 14:49:25.250376500 OK 2013-04-10 14:49:25.250380500 daily.cvd is up to date (version: 16985, sigs: 1079403, f-level: 63, builder: guitar) 2013-04-10 14:49:25.250449500 If-Modified-Since: Wed, 13 Feb 2013 15:29:15 GMT 2013-04-10 14:49:25.250449500 Reading CVD header (bytecode.cvd): Connected to clamav.datev.de (IP: 193.27.49.165). 2013-04-10 14:49:25.252128500 Trying to retrieve CVD header ofhttp://clamav.datev.de/bytecode.cvd 2013-04-10 14:49:25.253427500 OK 2013-04-10 14:49:25.253432500 bytecode.cvd is up to date (version: 214, sigs: 41, f-level: 63, builder: neo) 2013-04-10 14:49:25.253533500 LibClamAV debug: Initialized 0.97.7 engine 2013-04-10 14:49:25.253533500 LibClamAV debug: in cli_cvdload() 2013-04-10 14:49:25.253744500 LibClamAV debug: MD5(.tar.gz) = ff0c9c4e86dee9e705840537bd13ee71 2013-04-10 14:49:25.254423500 LibClamAV debug: cli_versig: Decoded signature: ff0c9c4e86dee9e705840537bd13ee71 2013-04-10 14:49:25.254424500 LibClamAV debug: cli_versig: Digital signature is correct. 2013-04-10 14:49:25.254435500 LibClamAV debug: in cli_tgzload() 2013-04-10 14:49:25.259901500 LibClamAV debug: bytecode.info loaded 2013-04-10 14:49:25.259913500 LibClamAV debug: in cli_tgzload() 2013-04-10 14:49:25.264665500 LibClamAV debug: Cleaning up phishcheck 2013-04-10 14:49:25.264666500 LibClamAV debug: Phishcheck cleaned up 2013-04-10 14:49:25.264699500 LibClamAV debug: Initialized 0.97.7 engine 2013-04-10 14:49:25.264699500 LibClamAV debug: in cli_cvdload() 2013-04-10 14:49:25.339354500 LibClamAV debug: MD5(.tar.gz) = ef015484e18b983ddf08425e2dad6a3f 2013-04-10 14:49:25.339964500 LibClamAV debug: cli_versig: Decoded signature: ef015484e18b983ddf08425e2dad6a3f 2013-04-10 14:49:25.339964500 LibClamAV debug: cli_versig: Digital signature is correct. 2013-04-10 14:49:25.339975500 LibClamAV debug: in cli_tgzload() 2013-04-10 14:49:25.345214500 LibClamAV debug: main.info loaded 2013-04-10 14:49:25.345219500 LibClamAV debug: in cli_tgzload() 2013-04-10 14:49:26.804849500 LibClamAV debug: Cleaning up phishcheck 2013-04-10 14:49:26.804850500 LibClamAV debug: Phishcheck cleaned up 2013-04-10 14:49:26.804873500 LibClamAV debug: Initialized 0.97.7 engine 2013-04-10 14:49:26.804874500 LibClamAV debug: in cli_cvdload() 2013-04-10 14:49:26.869060500 LibClamAV debug: MD5(.tar.gz) = da5da571c1a0369904550994d15dcc99 2013-04-10 14:49:26.869653500 LibClamAV debug: cli_versig: Decoded signature: da5da571c1a0369904550994d15dcc99 2013-04-10 14:49:26.869654500 LibClamAV debug: cli_versig: Digital signature is correct. 2013-04-10 14:49:26.869658500 LibClamAV debug: in cli_tgzload() 2013-04-10 14:49:26.875003500 LibClamAV debug: daily.info loaded 2013-04-10 14:49:26.875004500 LibClamAV debug: in cli_tgzload() 2013-04-10 14:49:28.147514500 LibClamAV debug: Cleaning up phishcheck 2013-04-10 14:49:28.147515500 LibClamAV debug: Phishcheck cleaned up 2013-04-10 14:49:28.147994500 0 Andreas ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] freshclam checks database every time
Hi-- On Apr 12, 2013, at 10:58 AM, Andreas Schulze wrote: But back to my main problem. clamscan wastes 3 secon[d]s time loading the complete engine every time it is called. Notice the timestamps! The data available to me suggests that ClamAV has seen ~736 database updates since the beginning of the year, for an average update frequency of ~3.5 hours between updates. ClamAV update process started at Tue Jan 1 01:07:36 2013 daily.cld updated (version: 16263, sigs: 465170, f-level: 63, builder: neo) Database updated (1509597 signatures) from db.us.clamav.net (IP: 150.214.142.197) ...vs... ClamAV update process started at Fri Apr 12 14:11:09 2013 daily.cld updated (version: 16999, sigs: 1092361, f-level: 63, builder: neo) Database updated (2136789 signatures) from db.us.clamav.net (IP: 69.163.100.14) Frankly, trying to update more often than once an hour is just a waste of resources. Regards, -- -Chuck ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] freshclam checks database every time
On 12.04.13 19:58, Andreas Schulze wrote: But back to my main problem. clamscan wastes 3 seconts time loading the complete engine every time it is called. loading virus databage to memory and parsing it is really CPU-intensive process. That's why we recommend running clamd daemon and using clamdscan. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. - Holmes, what kind of school did you study to be a detective? - Elementary, Watson. -- Daffy Duck Porky Pig ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] freshclam checks database every time
Am 10.04.2013 17:39, schrieb Al Varnell: I thought the limit was four times an hour. Is that only for mirror servers? cat /etc/cron.d/clamav * * * * * root freshclam :-) ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] freshclam checks database every time
On Thu, 2013-04-11 at 15:20 +0200, Andreas Schulze wrote: Am 10.04.2013 17:39, schrieb Al Varnell: I thought the limit was four times an hour. Is that only for mirror servers? cat /etc/cron.d/clamav * * * * * root freshclam :-) Is that checking your *OWN* mirror? If not, you are being throttled. -- greg folkert - systems administration and support web:donor.com email: g...@donor.com phone: 877-751-3300 x416 direct: 616-328-6449 (direct dial and fax) Faith has to do with things that are not seen and hope with things that are not at hand. -- Thomas Aquinas ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] freshclam checks database every time
On 4/10/13 6:05 AM, Andreas Schulze wrote: I configured freshclam to not lookup the dns for existance of a new patternversion. Instead freshclam contacts the clamav-server and fire up HTTP Head queries. That way I could let run freshclam once a minute. I thought the limit was four times an hour. Is that only for mirror servers? -Al- -- Al Varnell Mountain View, CA ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [Clamav-users] Freshclam checks.
El Lunes, 16 de Febrero de 2004 14:52, Tom Gwilt escribió: Check for a freshclam.conf file and check the settings there. Usually found in an /etc or /usr/local/etc directory. Yes here is the explanation: # How often check for a new database. We suggest checking for it every # two hours. Checks 12 Greetings. --- Carles Xavier Munyoz Baldó [EMAIL PROTECTED] http://www.unlimitedmail.net/ --- --- SF.Net is sponsored by: Speed Start Your Linux Apps Now. Build and deploy apps Web services for Linux with a free DVD software kit from IBM. Click Now! http://ads.osdn.com/?ad_id56alloc_id438op=click ___ Clamav-users mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/clamav-users
Re: [Clamav-users] Freshclam checks.
On Mon, 16 Feb 2004 at 13:38:27 +0100, Carles Xavier Munyoz Baldó wrote: Hi, I'm running freshcam in daemon mode and cheking for updates 2 times at day. I have seen in the log file that it is doing the check every 2 hours istead of 2 times at day. I launch the daemon this way: /internet/ClamAV/bin/freshclam -d --checks=2 --quiet -l /internet/ClamAV/log/freshclam.log What am I doing wrong ? Maybe you have also a cronjob which executes freshclam? -- Tomasz Papszun SysAdm @ TP S.A. Lodz, Poland | And it's only [EMAIL PROTECTED] http://www.lodz.tpsa.pl/ | ones and zeros. [EMAIL PROTECTED] http://www.ClamAV.net/ A GPL virus scanner --- SF.Net is sponsored by: Speed Start Your Linux Apps Now. Build and deploy apps Web services for Linux with a free DVD software kit from IBM. Click Now! http://ads.osdn.com/?ad_id=1356alloc_id=3438op=click ___ Clamav-users mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/clamav-users
Re: [Clamav-users] Freshclam checks.
On Mon, 16 Feb 2004 13:38:27 +0100 Carles Xavier Munyoz Bald [EMAIL PROTECTED] wrote: Hi, I'm running freshcam in daemon mode and cheking for updates 2 times at day. I have seen in the log file that it is doing the check every 2 hours istead of 2 times at day. I launch the daemon this way: /internet/ClamAV/bin/freshclam -d --checks=2 --quiet -l /internet/ClamAV/log/freshclam.log What am I doing wrong ? It seems freshclam is more responsible than you ;-) Best regards, Tomasz Kojm -- oo. [EMAIL PROTECTED] www.ClamAV.net (\/)\. http://www.clamav.net/gpg/tkojm.gpg \..._ 0DCA5A08407D5288279DB43454822DC8985A444B //\ /\ Mon Feb 16 14:37:50 CET 2004 pgp0.pgp Description: PGP signature
Re: [Clamav-users] Freshclam checks.
On Mon, 16 Feb 2004, Tomasz Papszun wrote: On Mon, 16 Feb 2004 at 13:38:27 +0100, Carles Xavier Munyoz Baldó wrote: Hi, I'm running freshcam in daemon mode and cheking for updates 2 times at day. I have seen in the log file that it is doing the check every 2 hours istead of 2 times at day. I launch the daemon this way: /internet/ClamAV/bin/freshclam -d --checks=2 --quiet -l /internet/ClamAV/log/freshclam.log What am I doing wrong ? Check for a freshclam.conf file and check the settings there. Usually found in an /etc or /usr/local/etc directory. Tom --- SF.Net is sponsored by: Speed Start Your Linux Apps Now. Build and deploy apps Web services for Linux with a free DVD software kit from IBM. Click Now! http://ads.osdn.com/?ad_id56alloc_id438op=click ___ Clamav-users mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/clamav-users
