Re: [clamav-users] Yet Another US Mirror Issue-Solved
On Sat, 17 Sep 2011 10:25:50 -0400 Dan dantear...@gmail.com wrote: At 1:33 PM +0200 9/16/2011, Tomasz Kojm wrote: On Thu, 15 Sep 2011 12:28:50 -0400 Dan dantear...@gmail.com wrote: At 10:43 AM +0200 9/15/2011, Tomasz Kojm wrote: OK, now please post the output of 'freshclam --list-mirrors' Mirror #9 IP: 88.198.67.125 Successes: 13 Failures: 0 Last access: Fri Aug 26 10:45:31 2011 Ignore: No - Mirror #10 IP: 65.19.179.67 Successes: 24 Failures: 5 Last access: Tue Sep 13 10:45:48 2011 Ignore: Yes Can't connect to port 80 of host database.clamav.net (IP: 88.198.67.125) is not considered a failure? Is there something that I can add to freshclam.conf to make it so? A connection problem was considered a failure in the past but it was making more harm than good. In most cases the problem lies at the user's end (keep in mind we have ~2M different IPs downloading the database every day) and according to our tests and user reports with the current settings freshclam can more effectively deal with network errors. So... there is nothing that an end-user can set that will make the connection error be considered a failure? I would need to go hack at the source code? I've enabled short-time blacklisting of mirrors on connection errors in clamav-devel, please give it a try (use the git version): http://www.clamav.net/lang/en/download/sources/ Does that Ignore: Yes entry automatically expire at some point or is that mirror now dead forever? They will expire automatically. The mirrors ignored for a short time should return to the pool after 30 minutes (or 6 hours in clamav-devel after recent changes), and those ignored for a long term (due to severe or repeating issues) should get cleared after 3 days. Regards, -- oo. Tomasz Kojm tk...@clamav.net (\/)\. http://www.ClamAV.net/gpg/tkojm.gpg \..._ 0DCA5A08407D5288279DB43454822DC8985A444B //\ /\ Mon Sep 19 13:50:14 CEST 2011 ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue-Solved
At 1:33 PM +0200 9/16/2011, Tomasz Kojm wrote: On Thu, 15 Sep 2011 12:28:50 -0400 Dan dantear...@gmail.com wrote: At 10:43 AM +0200 9/15/2011, Tomasz Kojm wrote: OK, now please post the output of 'freshclam --list-mirrors' Mirror #9 IP: 88.198.67.125 Successes: 13 Failures: 0 Last access: Fri Aug 26 10:45:31 2011 Ignore: No - Mirror #10 IP: 65.19.179.67 Successes: 24 Failures: 5 Last access: Tue Sep 13 10:45:48 2011 Ignore: Yes Can't connect to port 80 of host database.clamav.net (IP: 88.198.67.125) is not considered a failure? Is there something that I can add to freshclam.conf to make it so? A connection problem was considered a failure in the past but it was making more harm than good. In most cases the problem lies at the user's end (keep in mind we have ~2M different IPs downloading the database every day) and according to our tests and user reports with the current settings freshclam can more effectively deal with network errors. So... there is nothing that an end-user can set that will make the connection error be considered a failure? I would need to go hack at the source code? Does that Ignore: Yes entry automatically expire at some point or is that mirror now dead forever? I see that all but one of the mirrors marked as Ignore: Yes work fine (well, at least they respond to http://ip, which .125 does not). Is there a command I can throw at Clam that will reset that flag? Doesn't look like mirrors.dat is directly editable. How many connection failures are required before a mirror is taken out of rotation? Thu Sep 15 22:05:35 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Fri Sep 16 08:03:05 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Fri Sep 16 10:03:06 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Fri Sep 16 11:03:08 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Fri Sep 16 14:03:09 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Fri Sep 16 14:42:04 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Fri Sep 16 16:03:09 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Fri Sep 16 17:03:07 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Fri Sep 16 18:03:07 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Fri Sep 16 19:03:09 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Fri Sep 16 20:03:09 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Fri Sep 16 21:03:06 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Fri Sep 16 22:03:06 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Sat Sep 17 01:03:06 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Sat Sep 17 02:03:09 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Sat Sep 17 03:03:09 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Sat Sep 17 04:03:09 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Sat Sep 17 05:03:08 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Sat Sep 17 06:03:08 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Sat Sep 17 07:03:08 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Sat Sep 17 08:03:06 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Sat Sep 17 10:03:07 2011 - Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) That's a total of 0 successes over the past few days for that mirror. Thanks, - Dan. -- - Psychoceramic Emeritus; South Jersey, USA, Earth. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue-Solved
On Thu, 15 Sep 2011 12:38:40 -0700 Al Varnell alvarn...@mac.com wrote: [...] Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Database updated (1038850 signatures) from db.US.clamav.net (IP: 88.198.67.125) Database updated (1038850 signatures) from 88.198.67.125 (IP: 88.198.67.125) Database updated (1039253 signatures) from db.US.clamav.net (IP: 88.198.67.125) For all but 1 of the 21 of the Can't connects it immediately checked and connected and updated from another mirror, but the next update went right back to .125 in all but 2 cases. One Can't connect was followed by a second Can't connect. OK, so that's the expected behavior. -- oo. Tomasz Kojm tk...@clamav.net (\/)\. http://www.ClamAV.net/gpg/tkojm.gpg \..._ 0DCA5A08407D5288279DB43454822DC8985A444B //\ /\ Fri Sep 16 09:29:08 CEST 2011 ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue-Solved
On Thu, 15 Sep 2011 12:28:27 -0400 Dan dantear...@gmail.com wrote: At 10:42 PM -0700 9/14/2011, Al Varnell wrote: Against all odds I've had three updates in the last 24 and two of them have been from old .125 Not so lucky; here every freshclam run that has touched .125 includes a failure still. My latest: ClamAV update process started at Thu Sep 15 12:16:56 2011 Using IPv6 aware code Querying current.cvd.clamav.net TTL: 900 Software version from DNS: 0.97.2 main.cvd version from DNS: 53 main.cvd is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven) daily.cvd version from DNS: 13620 daily.cld is up to date (version: 13620, sigs: 193015, f-level: 60, builder: acab) safebrowsing.cvd version from DNS: 32299 Retrieving http://database.clamav.net/safebrowsing-32299.cdiff Ignoring mirror 65.19.179.67 (due to previous errors) nonblock_connect: connect timing out (30 secs) Can't connect to port 80 of host database.clamav.net (IP: 88.198.67.125) Ignoring mirror 65.19.179.67 (due to previous errors) Trying host database.clamav.net (207.57.106.31)... Trying to download http://database.clamav.net/safebrowsing-32299.cdiff (IP: 207.57.106.31) Downloading safebrowsing-32299.cdiff [100%] cdiff_apply: Parsed 56039 lines and executed 55991 commands Loading signatures from safebrowsing.cld Properly loaded 723320 signatures from new safebrowsing.cld safebrowsing.cld updated (version: 32299, sigs: 723320, f-level: 60, builder: google) Querying safebrowsing.32299.61.1.0.207.57.106.31.ping.clamav.net bytecode.cvd version from DNS: 144 bytecode.cld is up to date (version: 144, sigs: 41, f-level: 60, builder: edwin) Database updated (1762590 signatures) from database.clamav.net (IP: 207.57.106.31) This looks good, it properly switched to another mirror and successfully updated the database. -- oo. Tomasz Kojm tk...@clamav.net (\/)\. http://www.ClamAV.net/gpg/tkojm.gpg \..._ 0DCA5A08407D5288279DB43454822DC8985A444B //\ /\ Fri Sep 16 13:14:17 CEST 2011 ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue-Solved
On Thu, 15 Sep 2011 12:28:50 -0400 Dan dantear...@gmail.com wrote: At 10:43 AM +0200 9/15/2011, Tomasz Kojm wrote: OK, now please post the output of 'freshclam --list-mirrors' Mirror #9 IP: 88.198.67.125 Successes: 13 Failures: 0 Last access: Fri Aug 26 10:45:31 2011 Ignore: No - Mirror #10 IP: 65.19.179.67 Successes: 24 Failures: 5 Last access: Tue Sep 13 10:45:48 2011 Ignore: Yes Can't connect to port 80 of host database.clamav.net (IP: 88.198.67.125) is not considered a failure? Is there something that I can add to freshclam.conf to make it so? A connection problem was considered a failure in the past but it was making more harm than good. In most cases the problem lies at the user's end (keep in mind we have ~2M different IPs downloading the database every day) and according to our tests and user reports with the current settings freshclam can more effectively deal with network errors. Regards, -- oo. Tomasz Kojm tk...@clamav.net (\/)\. http://www.ClamAV.net/gpg/tkojm.gpg \..._ 0DCA5A08407D5288279DB43454822DC8985A444B //\ /\ Fri Sep 16 13:23:52 CEST 2011 ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue-Solved
On Wed, 14 Sep 2011 22:42:53 -0700 Al Varnell alvarn...@mac.com wrote: Against all odds I've had three updates in the last 24 and two of them have been from old .125, so I reserve the right to revisit the other part of the issue in a few days after I have some statistics on how often it gets used on the first attempt. Hey Al, please run 'freshclam -v' and look for this line: Using IPv6 aware code If it's not there, then most likely freshclam is using the older networking code, which does not randomize IP addresses on its own but only relies on the DNS. Regards, -- oo. Tomasz Kojm tk...@clamav.net (\/)\. http://www.ClamAV.net/gpg/tkojm.gpg \..._ 0DCA5A08407D5288279DB43454822DC8985A444B //\ /\ Thu Sep 15 10:17:13 CEST 2011 ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue-Solved
On 9/15/11 1:30 AM, Tomasz Kojm tk...@clamav.net wrote: Hey Al, please run 'freshclam -v' and look for this line: Using IPv6 aware code If it's not there, then most likely freshclam is using the older networking code, which does not randomize IP addresses on its own but only relies on the DNS. Here it is: Current working dir is /usr/local/clamXav/share/clamav Max retries == 3 ClamAV update process started at Thu Sep 15 01:37:17 2011 Using IPv6 aware code Querying current.cvd.clamav.net TTL: 224 Software version from DNS: 0.97.2 main.cvd version from DNS: 53 main.cld is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven) daily.cvd version from DNS: 13619 Retrieving http://db.US.clamav.net/daily-13618.cdiff Trying to download http://db.US.clamav.net/daily-13618.cdiff (IP: 207.57.106.31) Downloading daily-13618.cdiff [100%] cdiff_apply: Parsed 17 lines and executed 17 commands Retrieving http://db.US.clamav.net/daily-13619.cdiff Trying to download http://db.US.clamav.net/daily-13619.cdiff (IP: 207.57.106.31) Downloading daily-13619.cdiff [100%] cdiff_apply: Parsed 16 lines and executed 16 commands Loading signatures from daily.cld Properly loaded 193008 signatures from new daily.cld daily.cld updated (version: 13619, sigs: 193008, f-level: 60, builder: jesler) Querying daily.13619.62.1.0.207.57.106.31.ping.clamav.net bytecode.cvd version from DNS: 144 bytecode.cvd is up to date (version: 144, sigs: 41, f-level: 60, builder: edwin) Database updated (1039263 signatures) from db.US.clamav.net (IP: 207.57.106.31) Looks to be OK. -Al- -- Al Varnell Mountain View, CA ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue-Solved
On Thu, 15 Sep 2011 01:41:29 -0700 Al Varnell alvarn...@mac.com wrote: Looks to be OK. OK, now please post the output of 'freshclam --list-mirrors' -- oo. Tomasz Kojm tk...@clamav.net (\/)\. http://www.ClamAV.net/gpg/tkojm.gpg \..._ 0DCA5A08407D5288279DB43454822DC8985A444B //\ /\ Thu Sep 15 10:41:30 CEST 2011 ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue-Solved
On 9/15/11 1:43 AM, Tomasz Kojm tk...@clamav.net wrote: OK, now please post the output of 'freshclam --list-mirrors' Janets-iMac-G5:~ jvarnell$ sudo /usr/local/clamXav/bin/freshclam --list-mirrors Mirror #1 IP: 194.8.197.22 Successes: 12 Failures: 0 Last access: Tue Sep 13 15:45:14 2011 Ignore: No - Mirror #2 IP: 69.12.162.28 Successes: 4 Failures: 1 Last access: Thu Sep 8 07:45:14 2011 Ignore: No - Mirror #3 IP: 150.214.142.197 Successes: 6 Failures: 0 Last access: Wed Aug 24 07:45:08 2011 Ignore: No - Mirror #4 IP: 69.163.100.14 Successes: 13 Failures: 0 Last access: Tue Sep 13 19:40:16 2011 Ignore: No - Mirror #5 IP: 200.236.31.1 Successes: 8 Failures: 0 Last access: Thu Aug 25 15:45:11 2011 Ignore: No - Mirror #6 IP: 155.98.64.87 Successes: 14 Failures: 0 Last access: Wed Sep 7 15:45:09 2011 Ignore: No - Mirror #7 IP: 208.72.56.53 Successes: 7 Failures: 0 Last access: Sun Sep 11 07:45:04 2011 Ignore: No - Mirror #8 IP: 194.186.47.19 Successes: 6 Failures: 0 Last access: Fri Sep 9 15:45:19 2011 Ignore: No - Mirror #9 IP: 194.47.250.218 Successes: 31 Failures: 0 Last access: Sat Sep 10 15:47:31 2011 Ignore: No - Mirror #10 IP: 168.143.19.95 Successes: 13 Failures: 0 Last access: Mon Sep 5 07:45:07 2011 Ignore: No - Mirror #11 IP: 88.198.67.125 Successes: 10 Failures: 0 Last access: Wed Sep 14 15:46:40 2011 Ignore: No - Mirror #12 IP: 207.57.106.31 Successes: 6 Failures: 0 Last access: Thu Sep 15 01:37:23 2011 Ignore: No - Mirror #13 IP: 65.19.179.67 Successes: 8 Failures: 0 Last access: Sun Sep 11 15:47:38 2011 Ignore: No - Mirror #14 IP: 64.246.134.219 Successes: 8 Failures: 0 Last access: Tue Sep 13 07:45:07 2011 Ignore: No - Mirror #15 IP: 204.109.62.22 Successes: 5 Failures: 0 Last access: Thu Sep 8 15:45:17 2011 Ignore: No Note: Mirror #11 had Successes: 1 until yesterday. -Al- -- Al Varnell Mountain View, CA ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue-Solved
On Thu, 15 Sep 2011 02:11:16 -0700 Al Varnell alvarn...@mac.com wrote: [...] - Mirror #11 IP: 88.198.67.125 Successes: 10 Failures: 0 Last access: Wed Sep 14 15:46:40 2011 Ignore: No - Mirror #12 IP: 207.57.106.31 Successes: 6 Failures: 0 Last access: Thu Sep 15 01:37:23 2011 Ignore: No - Mirror #13 IP: 65.19.179.67 Successes: 8 Failures: 0 Last access: Sun Sep 11 15:47:38 2011 Ignore: No - Mirror #14 IP: 64.246.134.219 Successes: 8 Failures: 0 Last access: Tue Sep 13 07:45:07 2011 Ignore: No - Mirror #15 IP: 204.109.62.22 Successes: 5 Failures: 0 Last access: Thu Sep 8 15:45:17 2011 Ignore: No Note: Mirror #11 had Successes: 1 until yesterday. And that's the reason freshclam was choosing it as the first mirror all the time. Freshclam tries to balance the load by preferring mirrors with the lowest number of downloads. Then, when it fails to connect to such a mirror, it should disable this load balancing and simply pick up a random mirror in the next attempt. Could you check your logs to see if that actually happened? -- oo. Tomasz Kojm tk...@clamav.net (\/)\. http://www.ClamAV.net/gpg/tkojm.gpg \..._ 0DCA5A08407D5288279DB43454822DC8985A444B //\ /\ Thu Sep 15 14:07:40 CEST 2011 ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/14/2011 3:29 AM, sys...@ra-schaal.de wrote: i´ll have at look. but i´m moving until september to a new server with much bandwith (20 TB/month) and a better performance. maybe i can setup the mirror on this system on weekend. if you can´t connect to 88.198.67.125, you should fall back to 46.4.61.241. it seems, that freshclam won´t use the second ip. nslookup clamav.akxnet.de Server: 127.0.0.1 Address:127.0.0.1#53 Name: clamav.akxnet.de Address: 88.198.67.125 Name: clamav.akxnet.de Address: 46.4.61.241 if freshclam on one of my other servers tries connect to 88, i also can´t connect sometimes. but in this case freshclam just use the second ip (ie second server). i made some changes to the firewall. if it works be now, please mail me as soon as possible. That seems to have fixed the problem for me. I have seen three successful updates and no failures from your server since yesterday. Thanks. -- Bowie ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue-Solved
At 10:43 AM +0200 9/15/2011, Tomasz Kojm wrote: OK, now please post the output of 'freshclam --list-mirrors' Mirror #9 IP: 88.198.67.125 Successes: 13 Failures: 0 Last access: Fri Aug 26 10:45:31 2011 Ignore: No - Mirror #10 IP: 65.19.179.67 Successes: 24 Failures: 5 Last access: Tue Sep 13 10:45:48 2011 Ignore: Yes Can't connect to port 80 of host database.clamav.net (IP: 88.198.67.125) is not considered a failure? Is there something that I can add to freshclam.conf to make it so? - Dan. -- - Psychoceramic Emeritus; South Jersey, USA, Earth. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue-Solved
At 10:42 PM -0700 9/14/2011, Al Varnell wrote: Against all odds I've had three updates in the last 24 and two of them have been from old .125 Not so lucky; here every freshclam run that has touched .125 includes a failure still. My latest: ClamAV update process started at Thu Sep 15 12:16:56 2011 Using IPv6 aware code Querying current.cvd.clamav.net TTL: 900 Software version from DNS: 0.97.2 main.cvd version from DNS: 53 main.cvd is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven) daily.cvd version from DNS: 13620 daily.cld is up to date (version: 13620, sigs: 193015, f-level: 60, builder: acab) safebrowsing.cvd version from DNS: 32299 Retrieving http://database.clamav.net/safebrowsing-32299.cdiff Ignoring mirror 65.19.179.67 (due to previous errors) nonblock_connect: connect timing out (30 secs) Can't connect to port 80 of host database.clamav.net (IP: 88.198.67.125) Ignoring mirror 65.19.179.67 (due to previous errors) Trying host database.clamav.net (207.57.106.31)... Trying to download http://database.clamav.net/safebrowsing-32299.cdiff (IP: 207.57.106.31) Downloading safebrowsing-32299.cdiff [100%] cdiff_apply: Parsed 56039 lines and executed 55991 commands Loading signatures from safebrowsing.cld Properly loaded 723320 signatures from new safebrowsing.cld safebrowsing.cld updated (version: 32299, sigs: 723320, f-level: 60, builder: google) Querying safebrowsing.32299.61.1.0.207.57.106.31.ping.clamav.net bytecode.cvd version from DNS: 144 bytecode.cld is up to date (version: 144, sigs: 41, f-level: 60, builder: edwin) Database updated (1762590 signatures) from database.clamav.net (IP: 207.57.106.31) - Dan. -- - Psychoceramic Emeritus; South Jersey, USA, Earth. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue-Solved
On 9/15/11 5:19 AM, Tomasz Kojm tk...@clamav.net wrote: And that's the reason freshclam was choosing it as the first mirror all the time. Freshclam tries to balance the load by preferring mirrors with the lowest number of downloads. Then, when it fails to connect to such a mirror, it should disable this load balancing and simply pick up a random mirror in the next attempt. Could you check your logs to see if that actually happened? Here are the results of trying .125 over the same period, (Aug 29 to Sep 14). There were 42 attempted updates, 12 were already up-to-dates and 21 of 30 updates started with a 88.198.67.125 failure. Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host 88.198.67.125 (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Database updated (1038850 signatures) from db.US.clamav.net (IP: 88.198.67.125) Database updated (1038850 signatures) from 88.198.67.125 (IP: 88.198.67.125) Database updated (1039253 signatures) from db.US.clamav.net (IP: 88.198.67.125) For all but 1 of the 21 of the Can't connects it immediately checked and connected and updated from another mirror, but the next update went right back to .125 in all but 2 cases. One Can't connect was followed by a second Can't connect. -Al- -- Al Varnell Mountain View, CA ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/13/11 10:51 PM, Jim Preston wrote: Apple has chosen to go the Microsoft route of our users are too stupid to be allowed to do their own customization and as such we OS X users have to suffer as we do with the choices made in Redmond. I'm a Mac user for my personal workstation and I don't feel any such from the top notion. OS X itself is adequately self sufficient (for now) to allow us to skirt the nutters at Apple. Is it happens that OS X becomes more IOS centric then yes, thinkers are doomed. I do believe that Apple is heading away from the general purpose computer towards an Apple Store centric OS that must necessarily go ka-ching each time you wish something clever would run on your Mac. Some clues: VMware Fusion (hypervisor for Mac) will soon be an App-store only product. Same with Pixelmator (closest thing to photoshop for the Mac) and so for several others. And now we're way off topic, but it is true too for Windows users that all that we have grown up on is quickly ratcheting down to a single glass interface between us and our applications, and it is based on the iTunes model. I don't play gatekeeper well, so bumbye, general purpose computer Mac OS, hello Linux. SourceForge is my salvation. I hope. dp ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 09/13/2011 02:28 PM, Bryan Burke wrote: ...with zero successful connections to that IP. The connectivity failure is entirely reproducible by hand: % telnet 88.198.67.125 80 Trying 88.198.67.125... telnet: connect to address 88.198.67.125: Connection refused telnet: Unable to connect to remote host I should say that when I did this, I got the same, but the connection seemed to be timing out, not being refused (despite what telnet says). Was it the same for you? I ask because that would indicate either that the web server on that IP is down, or that some firewall is silently dropping packets. This is a good question. I had a problem with my ISP in that I could not access my work servers from home. My ISP stated that they were not filtering my work IPs and a traceroute seemed to confirm this. It seemed that some router along the way determined it did not like traffic with my originating IP and my office's terminating IP. Like many users, my IP address is not static but very rarely changes. Forcing an IP address change (method more OT then the rest of my post) from my ISP made all the traffic get through. I bring this up only because the symptoms are similar, some traffic like ICMP would make it fine while other traffic like HTTP and SSH were being blocked along the route. -- Jim Preston jimli...@commspeed.net ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On Sep 13, 2011, at 10:51 PM, Jim Preston jimli...@commspeed.net wrote: Well here I have to take exception. You have every option to choose mirrors that suit your liking better. If the US servers are not meeting your needs, pick a different region. If the US round-robin are using mirrors half way around the world, then. there is no detraction to picking default mirrors that are half way around the world but choosing something other then US as the location. The fact that ClamXav HAS chosen to make it inconvenient for users to change update frequency or setting of db mirrors is NOT a clamav fault. The mechanism exists in freschclam but the port to OS X has chosen to ignore this very important feature. Would you like me to write a user interface application so OS X users can do this very simple preference setting? And don't get me started on some of the stupid approaches Apple has taken to a very simple to manage OS like FreeBSD. Although I choose express no opinion on the MACH kernel versus other kernels, the MACH kernel choice, is not issu e that has detracted from the ability to easily set preferences. Apple has chosen to go the Microsoft route of our users are too stupid to be allowed to do their own customization and as such we OS X users have to suffer as we do with the choices made in Redmond. Thanks for the offer Jim. I realized after I hit send that I gave an emotional answer that I knew wasn't technically correct. As I think you know I try to represent the average ClamXav user here. In the forum I try very hard not to recommend solutions the average user won't be able to easily implement. If it's something that can be built into ClamXav I put it on Mark's suggested improvements list. Otherwise I may try it out myself, but rarely recommend tailoring clamav unless the user has special needs. As such, it's important that I maintain the baseline on my computer, otherwise I can't be as helpful to others. Believe it or not, I could care less whether my setup works for me or not as long as it performs the way it does for everybody else. Sent from Janet's iPad -Al- -- Al Varnell ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
Am 13.09.2011 20:49, schrieb Bowie Bailey: On 9/13/2011 1:18 PM, sys...@ra-schaal.de wrote: Am 13.09.2011 18:01, schrieb Al Varnell: On Sep 13, 2011, at 8:15 AM, Dan dantear...@gmail.com wrote: Yet more failure on 88.198.67.125, this morning. This one is a double. I was going to wait a few more days to mention this, but since you bring it up... I have seen this twice a day almost every day since 29 Aug. The only times I didn't see this was when the database was reported to be up-to-date. During that same period, I was _never_ able to successfully connect to it. This can't be just my bad luck. just your bad luck 2011/09/05 - 297638 connects 2011/09/06 - 265677 connects 2011/09/07 - 265228 connects 2011/09/08 - 210367 connects 2011/09/09 - 230462 connects 2011/09/10 - 142702 connects 2011/09/11 - 120486 connects 2011/09/12 - 207272 connects 2011/09/13 - 129521 connetcs until now - 1916 CET as mentioned a few days befor, YOU have a very slow connection to my system. Not just him. I don't hit your mirror every time, but the last time I was able to successfully update from it was Aug 28, which matches what Al reported. Since then, I have seen 23 errors: Can't connect to port 80 of host db.us.clamav.net (IP: 88.198.67.125) Trying it manually today, I can ping the server, but cannot connect to port 80. Seems like something changed on Aug 28 or 29 which is causing connection problems for some people. i´ll have at look. but i´m moving until september to a new server with much bandwith (20 TB/month) and a better performance. maybe i can setup the mirror on this system on weekend. if you can´t connect to 88.198.67.125, you should fall back to 46.4.61.241. it seems, that freshclam won´t use the second ip. nslookup clamav.akxnet.de Server: 127.0.0.1 Address:127.0.0.1#53 Name: clamav.akxnet.de Address: 88.198.67.125 Name: clamav.akxnet.de Address: 46.4.61.241 if freshclam on one of my other servers tries connect to 88, i also can´t connect sometimes. but in this case freshclam just use the second ip (ie second server). i made some changes to the firewall. if it works be now, please mail me as soon as possible. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/14/11 12:29 AM, sys...@ra-schaal.de sys...@ra-schaal.de wrote: i made some changes to the firewall. if it works be now, please mail me as soon as possible. I was able to connect via my browser. I forced an update by pulling out my bytecode.cld and it successfully downloaded it from your mirror. So whatever you did to the firewall seems to have solved at least that part of the problem. Now if I can just get freshclam to pick another mirror once in a while our problems should be over. -Al- -- Al Varnell Mountain View, CA ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 9/14/2011 2:29 AM, sys...@ra-schaal.de wrote: i made some changes to the firewall. if it works be now, please mail me as soon as possible. I started getting successful updates from 88.198.67.125 a couple hours after you posted this, and port 80 no longer shows closed from here. Thanks! -- Noel Jones -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOcKGiAAoJEJGRUHb5Oh6gLoMH/RnRPHpNfxpm8PTlkqh5sAtJ 6U9//hlV2Qinyq9zPjAX4RGUfMwXYWlTX3QnguWIsVkhEtfPC+kkdjq2S8KVNnpa VOQ1n0Ci5KaXifYK916jGjNKJ/AX6pAHcr6+I5jlzB5MO0IIfWTh7thPgaUfgIeK 49xd9gaMgwa+wW9VH96Qn18VYOLVbKdiRtUFBLdKdCzZt74HDdLw88e7nyWZJy0e NieuRTCsu0ib66ashU2uSgzoUpdDf84i874sQVGNFdNS6HRj4NyhgbeTTlSPsQ7j rcMXudLnwCHU/8rbQhWn2l+aT4idYrlWjyknZUVdBh16fqDmc/QF/kJYI/UVx7k= =qNDc -END PGP SIGNATURE- ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On Wed, 14 Sep 2011, Dan wrote: At 7:44 AM -0500 9/14/2011, Noel Jones wrote: On 9/14/2011 2:29 AM, sys...@ra-schaal.de wrote: i made some changes to the firewall. if it works be now, please mail me as soon as possible. I started getting successful updates from 88.198.67.125 a couple hours after you posted this, and port 80 no longer shows closed from here. Still not workin from here: http://www.downforeveryoneorjustme.com/88.198.67.125 Says it's up. == Chris Candreva -- ch...@westnet.com -- (914) 948-3162 WestNet Internet Services of Westchester http://www.westnet.com/ ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
Does your dig use the host table? Mine does not. Same with nslookup. I can't imagine why they would, in fact. Yea, I had to use getent hosts db.us.clamav.net to make sure the /etc/hosts entry was working. -- Bryan Burke IT Administrator Department of Electrical Engineering and Computer Science University of Tennessee, Knoxville bbu...@eecs.utk.edu (865) 974-4694 ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
At 7:44 AM -0500 9/14/2011, Noel Jones wrote: On 9/14/2011 2:29 AM, sys...@ra-schaal.de wrote: i made some changes to the firewall. if it works be now, please mail me as soon as possible. I started getting successful updates from 88.198.67.125 a couple hours after you posted this, and port 80 no longer shows closed from here. Still not workin from here: ClamAV update process started at Wed Sep 14 09:43:49 2011 main.cvd is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven) nonblock_connect: connect timing out (30 secs) Can't connect to port 80 of host database.clamav.net (IP: 88.198.67.125) Trying host database.clamav.net (207.57.106.31)... Downloading daily-13609.cdiff [100%] Downloading daily-13610.cdiff [100%] Downloading daily-13611.cdiff [100%] Downloading daily-13612.cdiff [100%] Downloading daily-13613.cdiff [100%] Downloading daily-13614.cdiff [100%] daily.cld updated (version: 13614, sigs: 192601, f-level: 60, builder: ccordes) bytecode.cld is up to date (version: 144, sigs: 41, f-level: 60, builder: edwin) Database updated (1038856 signatures) from database.clamav.net (IP: 207.57.106.31) Clamd successfully notified about the update. - Dan. -- - Psychoceramic Emeritus; South Jersey, USA, Earth. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On Wed, 14 Sep 2011, Dan wrote: http://www.downforeveryoneorjustme.com/88.198.67.125 Says it's up. Received responses: 53 Ok 5 Fail http://host-tracker.com/check_res_ajx/8730391-0/ Cheers, Steve Sanesecurity ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/13/2011 10:29 PM, Al Varnell wrote: I was trying to say that using this command: freshclam --stdout --quiet --no-warnings --log=/usr/local/clamXav/share/clamav/freshclam.log I can determine the IP address of a successful update in the last line, e.g. Database updated (1038839 signatures) from db.US.clamav.net (IP: 194.8.197.22) If the database is already up-to-date then there is no attempt to access a mirror, so it would not be possible to provide an IP. OOPS, my misunderstanding. :-) I didn't think there was really an issue, but figured I'd ask. freshclam logs what IP address it is about to try, reading what follows the try will usually tell you if it worked or not. -- Sincerely, Nathan Gibbs Systems Administrator Christ Media http://www.cmpublishers.com signature.asc Description: OpenPGP digital signature ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
At 12:36 PM -0400 9/13/2011, Bryan Burke wrote: Noone has suggested maximum. The issue is that the mirrors are so overloaded that it's often taking freshclam an excessive amount of time to do its thing, because of the time-outs / connection failures. No big deal if it's the update run in the background. But if it's on-demand update preceding a user-driven scan, it's making the user sit there, twiddling its thumbs, for up to a minute or two. Are we really having this protracted discussion, because we don't want someone to have to sit for up to a minute or two? This problem seems overstated. I mean, are we talking about on-demand scans perhaps a dozen or more times per day, every day? i.e. is this adding up to hours of lost time every week? If so, is it really such a problem to have a database that is *at most* 2 hours out-of-date (the default)? Do you need to do an update before *every* on-demand scan? And why can't that be solved (if it is, in fact, an issue) by increasing the check frequency to, say, every hour? Is it appropriate to ever do a scan against an outdated database? I've been told time and again never to do that! When a user launches their anti-virus app, they're going to want to check to see that their definitions are up-to-date. (I would argue that any app that doesn't force the update check by default is poorly designed). If that step takes a minute, instead of a few seconds, then the app becomes painful to use -- making them less likely to do scans in the future. Not good. Wanna make it worse? Put the user on a time-metered network connection! As for overstated... People that are both busy and security conscious tend to run quite a few scans per day. If each one halts their work for minutes... Or even if 1000 users have to wait that one minute just twice a day... then thats many hours wasted. And how many ClamAV users are there? (By user, in this context, I mean human at a desktop or laptop). *at most* 2 hours. Are you saying that freshclam should *always* be run in the background every hour or two *by everyone*, not just on servers? Can the current mirror infrastructure handle that? Currently, as a user app, ClamXav only runs freshclam in the background once per day, if the user enables such, but I'm sure we could get the author (Mark) to enhance its scheduling preferences. No big deal, IF that's the right thing to do. But even then... shouldn't every on-demand scan first do an update anyway??? (Running the update once per day isn't my fav design choice. Back in the day, when there were virtually no malwares for Mac OS X, I didn't have a problem with that. But these days, I think it needs to be fixed. Not an issue for this forum tho). At 3:49 PM -0400 9/13/2011, Bryan Burke wrote: I don't know the frequency, but it was enough of a problem for him to complain...three times before I brought it up here. So is this issue specifically with ClamXav? No. This is an issue specifically with *** freshclam *** and the reliability of *** ClamAV's Mirrors ***. I've seen the problem most often with ClamXav because me and mine use Macs. But I've received complaints about Clam from several of my clients recently - they use Clam on both their Macs and Windows machines. The update lag + the recent 2x not-updating-DNS has started the whole maybe it's time to evaluate other AV products cycle. Al wrote: Sending my browser to db.US.clamav.net gives me Safari can't open the page http://db.us.big.clamav.net/; because Safari can't connect to the server db.us.big.clamav.net. No matter how many times I try it. Ditto. Last night and this morning. The other mirrors respond quickly, but .125 - never. Just ran this: http://host-tracker.com/check_res_ajx/8730640-0/ and adding the results from this, previously in the thread: http://host-tracker.com/check_res_ajx/8730391-0/ It shows the average response time was under 3/4 of a second. Going down the lists, I see only a few sites took more than one second! So perhaps a time-out of 3 to 4 seconds would be more reasonable? 30s seems like painful overkill. At 6:15 PM -0400 9/13/2011, Bryan Burke wrote: If not, then at this point, I'm guessing there's enough data here for the team to make a decision one way or the other concerning this host. Even if removed, it can always be re-added when the cause of this issue is tracked down and fixed. At least concerning this issue, is there anything more to be done? 1) Fix freshclam so it doesn't stall for so long. 2) Fix freshclam so it doesn't ever use the same inaccessible mirror again, especially during the same run. 3) Get the unavailable mirror OUT of the rotation. - Dan. -- - Psychoceramic Emeritus; South Jersey, USA, Earth. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/14/11 12:29 AM, sys...@ra-schaal.de wrote: if you can´t connect to 88.198.67.125, you should fall back to 46.4.61.241. it seems, that freshclam won´t use the second ip. nslookup clamav.akxnet.de Server: 127.0.0.1 Address:127.0.0.1#53 Name: clamav.akxnet.de Address: 88.198.67.125 Name: clamav.akxnet.de Address: 46.4.61.241 Why would it? The client resolver has already identified 88.198.67.125 as the appropriate end point IP and won't ask again. If you are trying to use DNS at your end as a load balancer it isn't going to work. The expectation is that a reliable service is running only at the IP in the authorative DNS server at clamav.net and that IP is 88.198.67.125. Your depending on 46.4.61.241 as a fallback server is bad architecture. dp ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
At 12:36 PM -0400 9/13/2011, Bryan Burke wrote: Noone has suggested maximum. The issue is that the mirrors are so overloaded that it's often taking freshclam an excessive amount of time to do its thing, because of the time-outs / connection failures. No big deal if it's the update run in the background. But if it's on-demand update preceding a user-driven scan, it's making the user sit there, twiddling its thumbs, for up to a minute or two. Are we really having this protracted discussion, because we don't want someone to have to sit for up to a minute or two? This problem seems overstated. I mean, are we talking about on-demand scans perhaps a dozen or more times per day, every day? i.e. is this adding up to hours of lost time every week? If so, is it really such a problem to have a database that is *at most* 2 hours out-of-date (the default)? Do you need to do an update before *every* on-demand scan? And why can't that be solved (if it is, in fact, an issue) by increasing the check frequency to, say, every hour? Is it appropriate to ever do a scan against an outdated database? I've been told time and again never to do that! When a user launches their anti-virus app, they're going to want to check to see that their definitions are up-to-date. (I would argue that any app that doesn't force the update check by default is poorly designed). If that step takes a minute, instead of a few seconds, then the app becomes painful to use -- making them less likely to do scans in the future. Not good. Wanna make it worse? Put the user on a time-metered network connection! As for overstated... People that are both busy and security conscious tend to run quite a few scans per day. If each one halts their work for minutes... Or even if 1000 users have to wait that one minute just twice a day... then that's many hours wasted. And how many ClamAV users are there? (By user, in this context, I mean human at a desktop or laptop). *at most* 2 hours. Are you saying that freshclam should *always* be run in the background every hour or two *by everyone*, not just on servers? Can the current mirror infrastructure handle that? Currently, as a user app, ClamXav only runs freshclam in the background once per day, if the user enables such, but I'm sure we could get the author (Mark) to enhance its scheduling preferences. No big deal, IF that's the right thing to do. But even then... shouldn't every on-demand scan first do an update anyway??? (Running the update once per day isn't my fav design choice. Back in the day, when there were virtually no malwares for Mac OS X, I didn't have a problem with that. But these days, I think it needs to be fixed. Not an issue for this forum tho). At 3:49 PM -0400 9/13/2011, Bryan Burke wrote: I don't know the frequency, but it was enough of a problem for him to complain...three times before I brought it up here. So is this issue specifically with ClamXav? No. This is an issue specifically with *** freshclam *** and the reliability of *** ClamAV's Mirrors ***. I've seen the problem most often with ClamXav because me and mine use Macs. But I've received complaints about Clam from several of my clients recently - they use Clam on both their Macs and Windows machines. The update lag + the recent 2x not-updating-DNS has started the whole maybe it's time to evaluate other AV products cycle. Al wrote: Sending my browser to db.US.clamav.net gives me Safari can't open the page because Safari can't connect to the server db.us.big.clamav.net. No matter how many times I try it. Ditto. Last night and this morning. The other mirrors respond quickly, but .125 - never. Just ran this: http://host-tracker.com/check_res_ajx/8730640-0/ and adding the results from this, previously in the thread: http://host-tracker.com/check_res_ajx/8730391-0/ It shows the average response time was under 3/4 of a second. Going down the lists, I see only a few sites took more than one second! So perhaps a time-out of 3 to 4 seconds would be more reasonable? 30s seems like painful overkill. At 6:15 PM -0400 9/13/2011, Bryan Burke wrote: If not, then at this point, I'm guessing there's enough data here for the team to make a decision one way or the other concerning this host. Even if removed, it can always be re-added when the cause of this issue is tracked down and fixed. At least concerning this issue, is there anything more to be done? 1) Fix freshclam so it doesn't stall for so long. 2) Fix freshclam so it doesn't ever use the same inaccessible mirror again, especially during the same run. 3) Get the unavailable mirror OUT of the rotation. - Dan. -- - Psychoceramic Emeritus; South Jersey, USA, Earth. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
Am 14.09.2011 18:15, schrieb Dennis Peterson: expectation is that a reliable service is running only at the IP in the authorative DNS server at clamav.net and that IP is 88.198.67.125. Your depending on 46.4.61.241 as a fallback server is bad architecture. I told them a few months ago, and a few weeks ago, and for some reasons they didn´t add the second ip to the us-roundrobin on clamav.net. It´s only listed for russian-roundrobin. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
[clamav-users] Yet Another US Mirror Issue-Solved
It's been almost twenty-four hours since the Firewall was fixed, so I thought I'd take this opportunity to thank everybody involved, both for backing me up when there was much doubt and for offering useful suggestions and obviously to Florian for solving at his part of the issue. Hopefully US users (and probably others) will be a bit more productive because of it. Against all odds I've had three updates in the last 24 and two of them have been from old .125, so I reserve the right to revisit the other part of the issue in a few days after I have some statistics on how often it gets used on the first attempt. -Al- -- Al Varnell Mountain View, CA ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 09/14/2011 09:20 AM, Dan wrote: At 12:36 PM -0400 9/13/2011, Bryan Burke wrote: Noone has suggested maximum. The issue is that the mirrors are so overloaded that it's often taking freshclam an excessive amount of time to do its thing, because of the time-outs / connection failures. No big deal if it's the update run in the background. But if it's on-demand update preceding a user-driven scan, it's making the user sit there, twiddling its thumbs, for up to a minute or two. I do not recommend my users do their own scans. My recommendation is for scans to be scheduled to run during downtime such as at night or weekends. Are we really having this protracted discussion, because we don't want someone to have to sit for up to a minute or two? This problem seems overstated. I mean, are we talking about on-demand scans perhaps a dozen or more times per day, every day? i.e. is this adding up to hours of lost time every week? If so, is it really such a problem to have a database that is *at most* 2 hours out-of-date (the default)? Do you need to do an update before *every* on-demand scan? And why can't that be solved (if it is, in fact, an issue) by increasing the check frequency to, say, every hour? Is it appropriate to ever do a scan against an outdated database? I've been told time and again never to do that! This depends on whether it is an on-demand scan. If I have my AV set to do on-demand scanning (which I do have enabled for Windows because of the over whelming preference of virus writers to target Windows) then I ABSOLUTELY do not want the signatures to be updated everytime a scan is done. My Internet connection and the update servers would be overwhelmed by such aggressive updating I would think it would be considered an attack. On the other hand, if I suspect I have downloaded an infected file whether it be from the Internet, removable media, or LAN, then yes, I normally would want to be sure I had the latest signatures. Now this often involves a download to some other computer and a manual copy to the suspect computer as the first thing I do when I truly suspect I have managed to infect a system is to isolate it so it does not try and infect the rest of my network or worse start sending out replications tarnishing my reputation. In the case of routine scheduled scans of file systems, yes, I do not preferentially care if the signatures are several hours old. These scans are to see if there is a file that was not noted as infected earlier and is a preventative scan. If something suspicious turns up, then the previous paragraph applies. When a user launches their anti-virus app, they're going to want to check to see that their definitions are up-to-date. (I would argue that any app that doesn't force the update check by default is poorly designed). If that step takes a minute, instead of a few seconds, then the app becomes painful to use -- making them less likely to do scans in the future. Not good. Wanna make it worse? Put the user on a time-metered network connection! As for overstated... People that are both busy and security conscious tend to run quite a few scans per day. If each one halts their work for minutes... Or even if 1000 users have to wait that one minute just twice a day... then that's many hours wasted. And how many ClamAV users are there? (By user, in this context, I mean human at a desktop or laptop). *at most* 2 hours. Are you saying that freshclam should *always* be run in the background every hour or two *by everyone*, not just on servers? Can the current mirror infrastructure handle that? The answer on this is yes, every user should be updating their signatures every 2 hours which is why it is the freshclam default. If it is a work environment, then they should consider a local proxy server for the signatures to help reduce load on the mirrors. The mirrors should be scaled (and I believe they are) to handle a majority of the users to be directly downloading their own signatures. If they are security conscious then they should run them every hour. Currently, as a user app, ClamXav only runs freshclam in the background once per day, if the user enables such, but I'm sure we could get the author (Mark) to enhance its scheduling preferences. No big deal, IF that's the right thing to do. But even then... shouldn't every on-demand scan first do an update anyway??? (Running the update once per day isn't my fav design choice. Back in the day, when there were virtually no malwares for Mac OS X, I didn't have a problem with that. But these days, I think it needs to be fixed. Not an issue for this forum tho). Yes, ClamXav should have an easy to set preference to set the schedule. ClamXav is the first AV I have ever used where the user could not easily set the update schedule. The biggest danger is zero hour infections and running updates once a day is practically as bad as not bothering to
Re: [clamav-users] Yet Another US Mirror Issue
Yet more failure on 88.198.67.125, this morning. This one is a double. Shouldn't Freshclam be smart enough to avoid the same failing server at least within the same run? ClamAV update process started at Tue Sep 13 10:45:01 2011 main.cvd is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven) connect_error: getsockopt(SO_ERROR): fd=6 error=61: Connection refused Can't connect to port 80 of host database.clamav.net (IP: 88.198.67.125) Trying host database.clamav.net (65.19.179.67)... Downloading daily-13603.cdiff [100%] Downloading daily-13604.cdiff [100%] nonblock_recv: recv timing out (30 secs) connect_error: getsockopt(SO_ERROR): fd=6 error=61: Connection refused Can't connect to port 80 of host database.clamav.net (IP: 88.198.67.125) Trying host database.clamav.net (207.57.106.31)... Downloading daily-13605.cdiff [100%] Downloading daily-13606.cdiff [100%] Downloading daily-13607.cdiff [100%] Downloading daily-13608.cdiff [100%] daily.cld updated (version: 13608, sigs: 192488, f-level: 60, builder: neo) bytecode.cld is up to date (version: 144, sigs: 41, f-level: 60, builder: edwin) Database updated (1038743 signatures) from database.clamav.net (IP: 207.57.106.31) Clamd successfully notified about the update. - Dan. -- - Psychoceramic Emeritus; South Jersey, USA, Earth. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On Sep 13, 2011, at 8:15 AM, Dan dantear...@gmail.com wrote: Yet more failure on 88.198.67.125, this morning. This one is a double. I was going to wait a few more days to mention this, but since you bring it up... I have seen this twice a day almost every day since 29 Aug. The only times I didn't see this was when the database was reported to be up-to-date. During that same period, I was _never_ able to successfully connect to it. This can't be just my bad luck. Also, why was this mirror the first one checked from 2-10 Sep? I thought there was supposed to be more randomness in the list. This morning was the first time a different server appeared first this month. Sent from Janet's iPad -Al- -- Al Varnell ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
Noone has suggested maximum. The issue is that the mirrors are so overloaded that it's often taking freshclam an excessive amount of time to do its thing, because of the time-outs / connection failures. No big deal if it's the update run in the background. But if it's on-demand update preceding a user-driven scan, it's making the user sit there, twiddling its thumbs, for up to a minute or two. Are we really having this protracted discussion, because we don't want someone to have to sit for up to a minute or two? This problem seems overstated. I mean, are we talking about on-demand scans perhaps a dozen or more times per day, every day? i.e. is this adding up to hours of lost time every week? If so, is it really such a problem to have a database that is *at most* 2 hours out-of-date (the default)? Do you need to do an update before *every* on-demand scan? And why can't that be solved (if it is, in fact, an issue) by increasing the check frequency to, say, every hour? I'm not trying to stifle the idea of distributing the databases via torrent, but some of this discussion seems to be trying to solve a fabricated issue. As for the torrent, I think we can stop the discussion given the following: 1. The ClamAV team has said they will not support torrents. 2. The question about the local directory has been addressed. 3. Torrents can be easily created by anyone. Is there really anything more to discuss, except perhaps some more details of the local directory answer? -- Bryan Burke IT Administrator Department of Electrical Engineering and Computer Science University of Tennessee, Knoxville bbu...@eecs.utk.edu (865) 974-4694 ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
Am 13.09.2011 18:01, schrieb Al Varnell: On Sep 13, 2011, at 8:15 AM, Dan dantear...@gmail.com wrote: Yet more failure on 88.198.67.125, this morning. This one is a double. I was going to wait a few more days to mention this, but since you bring it up... I have seen this twice a day almost every day since 29 Aug. The only times I didn't see this was when the database was reported to be up-to-date. During that same period, I was _never_ able to successfully connect to it. This can't be just my bad luck. just your bad luck 2011/09/05 - 297638 connects 2011/09/06 - 265677 connects 2011/09/07 - 265228 connects 2011/09/08 - 210367 connects 2011/09/09 - 230462 connects 2011/09/10 - 142702 connects 2011/09/11 - 120486 connects 2011/09/12 - 207272 connects 2011/09/13 - 129521 connetcs until now - 1916 CET as mentioned a few days befor, YOU have a very slow connection to my system. just use another mirror instead of crying all the time about your bad setup. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/13/2011 1:18 PM, sys...@ra-schaal.de wrote: Am 13.09.2011 18:01, schrieb Al Varnell: On Sep 13, 2011, at 8:15 AM, Dan dantear...@gmail.com wrote: Yet more failure on 88.198.67.125, this morning. This one is a double. I was going to wait a few more days to mention this, but since you bring it up... I have seen this twice a day almost every day since 29 Aug. The only times I didn't see this was when the database was reported to be up-to-date. During that same period, I was _never_ able to successfully connect to it. This can't be just my bad luck. just your bad luck 2011/09/05 - 297638 connects 2011/09/06 - 265677 connects 2011/09/07 - 265228 connects 2011/09/08 - 210367 connects 2011/09/09 - 230462 connects 2011/09/10 - 142702 connects 2011/09/11 - 120486 connects 2011/09/12 - 207272 connects 2011/09/13 - 129521 connetcs until now - 1916 CET as mentioned a few days befor, YOU have a very slow connection to my system. Not just him. I don't hit your mirror every time, but the last time I was able to successfully update from it was Aug 28, which matches what Al reported. Since then, I have seen 23 errors: Can't connect to port 80 of host db.us.clamav.net (IP: 88.198.67.125) Trying it manually today, I can ping the server, but cannot connect to port 80. Seems like something changed on Aug 28 or 29 which is causing connection problems for some people. -- Bowie ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/13/11 9:36 AM, Bryan Burke bbu...@eecs.utk.edu wrote: Noone has suggested maximum. The issue is that the mirrors are so overloaded that it's often taking freshclam an excessive amount of time to do its thing, because of the time-outs / connection failures. No big deal if it's the update run in the background. But if it's on-demand update preceding a user-driven scan, it's making the user sit there, twiddling its thumbs, for up to a minute or two. Are we really having this protracted discussion, because we don't want someone to have to sit for up to a minute or two? That was the original intent, but we seem to have hit a couple of other nerves. This problem seems overstated. I mean, are we talking about on-demand scans perhaps a dozen or more times per day, every day? i.e. is this adding up to hours of lost time every week? If so, is it really such a problem to have a database that is *at most* 2 hours out-of-date (the default)? Do you need to do an update before *every* on-demand scan? I don't know the frequency, but it was enough of a problem for him to complain...three times before I brought it up here. And why can't that be solved (if it is, in fact, an issue) by increasing the check frequency to, say, every hour? That's not a user option with ClamXav, although I realize it could be done by hacking the LaunchAgent (formerly cron) event. I will probably recommend to Mark that he include multiple updates as a user preference one of these days, but there are a couple of other features I'd like to see first. ... Is there really anything more to discuss, except perhaps some more details of the local directory answer? As I mentioned earlier today, I believe the issue with this particular mirror is bigger than what has been stated. I understand the need to limit access but why do we have a mirror: - Supporting users half way around the world - Which always seems to be the first one checked - And has never successfully connected for over two weeks If it was just one of these I could accept it, but there has to be something else going on with it. My guess is that if the network was working as designed the user would never had lodged his initial complaint. -Al- -- Al Varnell Mountain View, CA ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/13/11 10:18 AM, sys...@ra-schaal.de sys...@ra-schaal.de wrote: Am 13.09.2011 18:01, schrieb Al Varnell: On Sep 13, 2011, at 8:15 AM, Dan dantear...@gmail.com wrote: Yet more failure on 88.198.67.125, this morning. This one is a double. I was going to wait a few more days to mention this, but since you bring it up... I have seen this twice a day almost every day since 29 Aug. The only times I didn't see this was when the database was reported to be up-to-date. During that same period, I was _never_ able to successfully connect to it. This can't be just my bad luck. just your bad luck 2011/09/05 - 297638 connects 2011/09/06 - 265677 connects 2011/09/07 - 265228 connects 2011/09/08 - 210367 connects 2011/09/09 - 230462 connects 2011/09/10 - 142702 connects 2011/09/11 - 120486 connects 2011/09/12 - 207272 connects 2011/09/13 - 129521 connetcs until now - 1916 CET as mentioned a few days befor, YOU have a very slow connection to my system. I'm half a world away from you, so I'm not really surprised by that, but what difference should it make? just use another mirror instead of crying all the time about your bad setup. What are you talking about? I have no choice whatsoever on the mirror I connect to! -Al- -- Al Varnell Mountain View, CA ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
I don't know the frequency, but it was enough of a problem for him to complain...three times before I brought it up here. So is this issue specifically with ClamXav? i.e. is ClamXav forcing an update each time it's run? I know that the regular clamav does not do this, and if that's the product in question, my point still seems valid: aren't we crying over spilled milk here? I mean, it would seem that the user's desired case/functionality is unreasonable, and as a result, that asking the ClamAV team to do anything about it is also unreasonable. If the issue is, however, with ClamXav, then this isn't the correct mailing list to be having this discussion, correct? That's not a user option with ClamXav, although I realize it could be done by hacking the LaunchAgent (formerly cron) event. I will probably recommend to Mark that he include multiple updates as a user preference one of these days, but there are a couple of other features I'd like to see first. Fair enough. - Supporting users half way around the world Don't see a problem with this. - Which always seems to be the first one checked Actual issue. Perhaps DNS caching is a factor? If freshclam checks often enough, then perhaps the cache entry never dies, and you get the same order every time? - And has never successfully connected for over two weeks Other than an announcement to the list that there may be problems with one of the mirrors, this seems to be an issue primarily between those users who encountered said error (and caused them distress) and the mirror admins, not the whole list. However, maybe I'm wrong and many readers of the list appreciate seeing the back-and-forth. P.S. - My goal is to try to limit the scope of this thread a little more, so it stays focused and relevant. As a side-line user on this list, I feel it had long since gotten out-of-hand. -- Bryan Burke IT Administrator Department of Electrical Engineering and Computer Science University of Tennessee, Knoxville bbu...@eecs.utk.edu (865) 974-4694 ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
Hi-- On Sep 13, 2011, at 12:49 PM, Bryan Burke wrote: - Which always seems to be the first one checked Actual issue. Perhaps DNS caching is a factor? If freshclam checks often enough, then perhaps the cache entry never dies, and you get the same order every time? Running dig db.us.clamav.net a few times shows that the nameserver responses are rotating the resource records; and even if it didn't, well-behaved resolver clients ought to rotate through multiple valid IPs returned by gethostbyname()/getaddrinfo() for a hostname anyway. - And has never successfully connected for over two weeks Other than an announcement to the list that there may be problems with one of the mirrors, this seems to be an issue primarily between those users who encountered said error (and caused them distress) and the mirror admins, not the whole list. However, maybe I'm wrong and many readers of the list appreciate seeing the back-and-forth. P.S. - My goal is to try to limit the scope of this thread a little more, so it stays focused and relevant. As a side-line user on this list, I feel it had long since gotten out-of-hand. I admire your goal of focussing on the problem, which I why I'll reply to this rather than other emails. :-) This being said, there is definitely a recurring issue with this particular mirror. Since Aug 22, I've seen: % grep Can't connect to port 80 of host database.clamav.net (IP: 88.198.67.125) /var/log/freshclam.log | wc -l 27 ...with zero successful connections to that IP. The connectivity failure is entirely reproducible by hand: % telnet 88.198.67.125 80 Trying 88.198.67.125... telnet: connect to address 88.198.67.125: Connection refused telnet: Unable to connect to remote host I don't consider this to be a significant problem since other mirrors are up, but it's not a matter of bandwidth or connectivity on my side. As it happens, I'm testing from Cupertino, CA via Apple's 17.0.0.0/8 network, and from a Time-Warner cable link from NYC, NY on 24.103.0.0/16. However, as a workaround it should be possible for folks to manually set DatabaseMirror in freshclam.conf to specific IPs from db.us.clamav.net, or perhaps switch to using db.ca.clamav.net, db.mx.clamav.net, or similar. Regards, -- -Chuck ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On Sep 13, 2011, at 2:28 PM, Bryan Burke wrote: ...with zero successful connections to that IP. The connectivity failure is entirely reproducible by hand: % telnet 88.198.67.125 80 Trying 88.198.67.125... telnet: connect to address 88.198.67.125: Connection refused telnet: Unable to connect to remote host I should say that when I did this, I got the same, but the connection seemed to be timing out, not being refused (despite what telnet says). Was it the same for you? No, I get an immediate connection refused and an ICMP port unreachable back: # tcpdump -nq host 88.198.67.125 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on en0, link-type EN10MB (Ethernet), capture size 65535 bytes 14:32:31.222347 IP 17.209.4.71.55899 88.198.67.125.80: tcp 0 14:32:31.397480 IP 88.198.67.125 17.209.4.71: ICMP 88.198.67.125 tcp port 80 unreachable, length 72 ^C 2 packets captured I ask because that would indicate either that the web server on that IP is down, or that some firewall is silently dropping packets. The webserver appears down from here; while a firewall could be configured to return ICMP_UNREACH_PORT, normally they just drop the traffic and you get connection timeouts as you've described... Regards, -- -Chuck ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/13/2011 12:47 AM, Henrik K wrote: If you are an individual not able to put $15-$100 a month, then yes, it's not in your capability. $15 - $100 extra / month would go to higher priority tasks / needs. Some of our servers are nearly old enough to vote. :-) As an individual, or small company, it just isn't within our current capabilities. When it is, we plan to get involved there. For now, we do what we can with what we have. No one thinks any less of you for trying to help, on the contrary. But if you can't even get any facts straight etc, it's just messing up the thread. Let's not forget that ClamAV is backed by a commercial organization?? If they wanted US bandwidth badly, they can get it. If not by buying, then probably just by asking around or even on the web page? Why do you think it's not mentioned there. Probably very few users read this list. Very good point. They could get it if they really needed it. Asking the user base for it is kind of sad. -- Sincerely, Nathan Gibbs Systems Administrator Christ Media http://www.cmpublishers.com signature.asc Description: OpenPGP digital signature ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
No, I get an immediate connection refused and an ICMP port unreachable back: # tcpdump -nq host 88.198.67.125 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on en0, link-type EN10MB (Ethernet), capture size 65535 bytes 14:32:31.222347 IP 17.209.4.71.55899 88.198.67.125.80: tcp 0 14:32:31.397480 IP 88.198.67.125 17.209.4.71: ICMP 88.198.67.125 tcp port 80 unreachable, length 72 My fault; just different telnet behaviors: I was using BSD telnet, which apparently kept trying to connect. When I used linux telnet, it ends immediately. So no discrepancy there. And I momentarily forgot the behavior of so-called closed ports (not blocked by firewall, but nothing running on them... thought the packets were dropped). So assuming a common firewall setup, it would appear the webserver is down. For potential aid in comparing notes and diagnosing the problem, I'm attaching some network information (whois and traceroute). If no firewall rule at the remote site explains this, then I can only surmise that some hop along the way is blocking the connections. If, however, this is due to some rate-limiting rule at the end point, is that acceptable? I don't know if ClamAV has a policy they ask their mirror hosts to adhere to, but if so, would this constitute grounds for removal from the pool? If not, then at this point, I'm guessing there's enough data here for the team to make a decision one way or the other concerning this host. Even if removed, it can always be re-added when the cause of this issue is tracked down and fixed. At least concerning this issue, is there anything more to be done? -- Bryan Burke IT Administrator Department of Electrical Engineering and Computer Science University of Tennessee, Knoxville bbu...@eecs.utk.edu (865) 974-4694 WHOIS: The University of Tennessee Health Science Center UTK-NET (NET-160-36-0-0-1) 160.36.0.0 - 160.36.255.255 Various Registries (Maintained by ARIN) NET160 (NET-160-0-0-0-0) 160.0.0.0 - 160.255.255.255 traceroute: 1 chm01v150.ns.utk.edu (160.36.56.1) 0.383 ms 0.430 ms 0.371 ms 2 10.8.2.30 (10.8.2.30) 0.605 ms 0.547 ms 0.477 ms 3 bsm01v20.ns.utk.edu (160.36.128.133) 0.962 ms 0.967 ms 0.975 ms 4 bhm01ge3-3.ns.utk.edu (160.36.2.74) 0.671 ms 0.940 ms 0.869 ms 5 gi1-8.mpd01.atl04.atlas.cogentco.com (38.104.182.37) 6.564 ms 6.551 ms 6.580 ms 6 te0-1-0-1.mpd22.atl01.atlas.cogentco.com (154.54.3.169) 18.520 ms te0-1-0-1.ccr22.atl01.atlas.cogentco.com (154.54.6.121) 18.685 ms 18.603 ms 7 te0-4-0-7.mpd22.dca01.atlas.cogentco.com (154.54.27.93) 18.552 ms te0-1-0-2.ccr22.dca01.atlas.cogentco.com (154.54.28.230) 18.521 ms te0-2-0-3.mpd22.dca01.atlas.cogentco.com (154.54.2.102) 18.642 ms 8 te0-1-0-1.ccr22.iad02.atlas.cogentco.com (154.54.26.138) 19.529 ms te0-1-0-1.mpd22.iad02.atlas.cogentco.com (154.54.26.122) 19.656 ms te0-3-0-5.ccr22.iad02.atlas.cogentco.com (154.54.41.238) 19.922 ms 9 te1-8.ccr02.iad01.atlas.cogentco.com (154.54.31.174) 19.450 ms te2-7.ccr02.iad01.atlas.cogentco.com (154.54.31.214) 19.676 ms te1-2.ccr02.iad01.atlas.cogentco.com (154.54.31.194) 19.713 ms 10 kpn.iad01.atlas.cogentco.com (154.54.10.242) 19.364 ms 19.434 ms 19.377 ms 11 nyk-s2-rou-1021.US.eurorings.net (134.222.227.133) 26.53 ms 25.576 ms 25.506 ms 12 nntr-s1-rou-1022.FR.eurorings.net (134.222.226.162) 101.182 ms 103.179 ms 101.83 ms 13 ffm-s1-rou-1022.DE.eurorings.net (134.222.229.30) 117.550 ms 117.294 ms 117.393 ms 14 ffm-s1-rou-1021.DE.eurorings.net (134.222.228.85) 118.820 ms 116.595 ms 118.851 ms 15 nbg-s1-rou-1001.DE.eurorings.net (134.222.225.26) 119.864 ms 120.319 ms 120.34 ms 16 kpn-gw.hetzner.de (134.222.107.21) 121.689 ms 121.654 ms 121.642 ms 17 hos-bb2.juniper1.fs.hetzner.de (213.239.240.146) 122.426 ms hos-bb2.juniper2.rz14.hetzner.de (213.239.240.151) 123.412 ms 123.453 ms 18 hos-tr2.ex3k4.rz14.hetzner.de (213.239.224.165) 124.146 ms hos-tr1.ex3k4.rz14.hetzner.de (213.239.224.133) 128.706 ms 127.250 ms 19 mx00.akxnet.de (88.198.67.125) 122.800 ms 122.781 ms 122.707 ms traceroute -n: 1 160.36.56.1 0.456 ms 2.169 ms 2.226 ms 2 10.8.2.30 7.586 ms 0.622 ms 0.563 ms 3 160.36.128.133 0.541 ms 0.529 ms 0.566 ms 4 160.36.2.74 0.594 ms 0.580 ms 0.630 ms 5 38.104.182.37 6.674 ms 6.600 ms 6.551 ms 6 154.54.3.169 18.612 ms 154.54.6.121 18.850 ms 19.305 ms 7 154.54.3.66 18.513 ms 154.54.1.122 18.616 ms 154.54.27.97 18.489 ms 8 154.54.30.126 19.643 ms 154.54.30.118 19.548 ms 154.54.7.158 19.570 ms 9 154.54.31.214 19.513 ms 154.54.31.174 19.478 ms 154.54.31.234 19.504 ms 10 154.54.10.242 19.359 ms 19.324 ms 19.288 ms 11 134.222.227.133 42.719 ms 33.734 ms 32.88 ms 12 134.222.226.162 101.309 ms 101.216 ms 112.846 ms 13 134.222.231.145 118.146 ms 118.101 ms 118.99 ms 14 134.222.228.89 120.349 ms 118.313 ms 124.437 ms 15 134.222.225.26 119.494 ms 119.264 ms 119.573 ms 16
Re: [clamav-users] Yet Another US Mirror Issue
On 9/13/11 12:49 PM, Bryan Burke bbu...@eecs.utk.edu wrote: I don't know the frequency, but it was enough of a problem for him to complain...three times before I brought it up here. So is this issue specifically with ClamXav? i.e. is ClamXav forcing an update each time it's run? No, the option to check updates at launch defaults to off, but this particular user prefers to have the most recent updates available when running manual checks, so he has toggled the option on. I know that the regular clamav does not do this, and if that's the product in question, my point still seems valid: aren't we crying over spilled milk here? I mean, it would seem that the user's desired case/functionality is unreasonable, and as a result, that asking the ClamAV team to do anything about it is also unreasonable. If the issue is, however, with ClamXav, then this isn't the correct mailing list to be having this discussion, correct? Correct and it has been extensively discussed on the ClamXav Forum long before I brought it here. He has tried all the suggestions we made and still feels like he's wasting a log of time. If one user isn't enough to justify making any changes, fair enough, but I firmly believe we have a systemic problem that affects all US users here that needs to be resolved. That's not a user option with ClamXav, although I realize it could be done by hacking the LaunchAgent (formerly cron) event. I will probably recommend to Mark that he include multiple updates as a user preference one of these days, but there are a couple of other features I'd like to see first. Fair enough. - Supporting users half way around the world Don't see a problem with this. Not under normal circumstances, but from the Traceroutes I and others have done there does seem to be a significant delay in the Trans Atlantic segment. If that's what's causing the failure to connects, then maybe we need to take a look at the viability of where we go for off-shore mirrors. - Which always seems to be the first one checked Actual issue. Perhaps DNS caching is a factor? If freshclam checks often enough, then perhaps the cache entry never dies, and you get the same order every time? Interesting thought. - And has never successfully connected for over two weeks Other than an announcement to the list that there may be problems with one of the mirrors, this seems to be an issue primarily between those users who encountered said error (and caused them distress) and the mirror admins, not the whole list. However, maybe I'm wrong and many readers of the list appreciate seeing the back-and-forth. I'm more than willing to take this off-line if someone can give me a list of everybody that needs to be part of the discussion. P.S. - My goal is to try to limit the scope of this thread a little more, so it stays focused and relevant. As a side-line user on this list, I feel it had long since gotten out-of-hand. -Al- -- Al Varnell Mountain View, CA ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/13/11 2:28 PM, Bryan Burke bbu...@eecs.utk.edu wrote: % grep Can't connect to port 80 of host database.clamav.net (IP: 88.198.67.125) /var/log/freshclam.log | wc -l 27 Interesting. When I just grep for the IP in my logs: ib /var/log # grep 88.198.67.125 maillog* | wc -l 12 ren /var/log # grep 88.198.67.125 maillog* | wc -l 5 ba /var/log # grep 88.198.67.125 maillog* | wc -l 12 That represents 7 days worth of logs, across three servers. That averages to ~10/day. Note that my systems are configured for the default, which is 12 DB update checks per day. Since freshclam doesn't seem to log the IP (by default, at least) when the update succeeds (or there is no update), I have no good way of checking how many times 88.198.67.125 is queried. My logs show successful update sources in the last line, but not when there is no update. For instance, here is the one that just occurred: -- ClamAV update process started at Tue Sep 13 15:45:07 2011 main.cld is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven) connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Downloading daily-13609.cdiff [100%] daily.cld updated (version: 13609, sigs: 192584, f-level: 60, builder: neo) bytecode.cld is up to date (version: 144, sigs: 41, f-level: 60, builder: edwin) Database updated (1038839 signatures) from db.US.clamav.net (IP: 194.8.197.22) -- -Al- -- Al Varnell Mountain View, CA ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/13/2011 7:07 PM, Al Varnell wrote: On 9/13/11 2:28 PM, Bryan Burke bbu...@eecs.utk.edu wrote: Since freshclam doesn't seem to log the IP (by default, at least) when the update succeeds (or there is no update), I have no good way of checking how many times 88.198.67.125 is queried. My logs show successful update sources in the last line, but not when there is no update. Which log messages need the IP? I'm testing the next CCEE patch set, so I could possibly slip those changes in before release. :-) -- Sincerely, Nathan Gibbs Systems Administrator Christ Media http://www.cmpublishers.com signature.asc Description: OpenPGP digital signature ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
My logs show successful update sources in the last line, but not when there is no update. Ok, well I did check the output of the grep before posting the number of lines on this list, and all log entries mentioning that IP were failures. So there's still *technically* some gray area, in that, if it happened to query that IP successfully, and there was no update, we'd never know, but I'm guessing that would reveal a similar outcome. Another side note: My ping latency times were about half of those posted earlier in the thread and I can't connect (about 122ms average). Either way, I really doubt the high latency of 250ms would cause any sort of issue. -- Bryan Burke IT Administrator Department of Electrical Engineering and Computer Science University of Tennessee, Knoxville bbu...@eecs.utk.edu (865) 974-4694 ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/13/11 6:31 PM, Nathan Gibbs nat...@cmpublishers.com wrote: On 9/13/2011 7:07 PM, Al Varnell wrote: On 9/13/11 2:28 PM, Bryan Burke bbu...@eecs.utk.edu wrote: Since freshclam doesn't seem to log the IP (by default, at least) when the update succeeds (or there is no update), I have no good way of checking how many times 88.198.67.125 is queried. My logs show successful update sources in the last line, but not when there is no update. Which log messages need the IP? I was trying to say that using this command: freshclam --stdout --quiet --no-warnings --log=/usr/local/clamXav/share/clamav/freshclam.log I can determine the IP address of a successful update in the last line, e.g. Database updated (1038839 signatures) from db.US.clamav.net (IP: 194.8.197.22) If the database is already up-to-date then there is no attempt to access a mirror, so it would not be possible to provide an IP. But appreciate the offer. -Al- -- Al Varnell Mountain View, CA ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
Eliminate some unknowns - like maybe your DNS doesn't like big packets. Add this *temporarily* to your host table: 88.198.67.125 db.us.big.clamav.net And try again - and try with your browser, too. It should show you a web page indentifying the site you connected to and then after a short time you will be sent to clamav.net. Running host db.us.big.clamav.net multiple times seems to reveal 15 servers in the pool, and the order changes each time; as I mentioned earlier, in my case at least, the random pool idea is working, even if over a 7-day period, 1/7 attempts to update seemed to try the IP in question... just the nature of randomness, I suppose. Also, how would this reveal anything more than what telnet 88.198.67.125 80 getting a connection refused tells us? However, I did just discover something bizarre and interesting: telnet 88.198.67.125 80 Trying 88.198.67.125... telnet: connect to address 88.198.67.125: Connection refused host 88.198.67.125 125.67.198.88.in-addr.arpa domain name pointer mx00.akxnet.de. host mx00.akxnet.de mx00.akxnet.de has address 88.198.67.99 mx00.akxnet.de has IPv6 address 2a01:4f8:140:4301::2 telnet 88.198.67.99 80 Trying 88.198.67.99... Connected to 88.198.67.99. Escape character is '^]'. ^] telnet quit Connection closed. Is it possible this is caused by a master DNS issue? Of course, I tried to see the page and didn't get much, but I'm not all that familiar with HTTP: curl -H Host: db.us.clamav.net 88.198.67.99 !DOCTYPE HTML PUBLIC -//IETF//DTD HTML 2.0//EN htmlhead title403 Forbidden/title /headbody h1Forbidden/h1 pYou don't have permission to access / on this server./p hr addressApache/2.2.15 (Linux/SUSE) Server at db.us.clamav.net Port 80/address /body/html -- Bryan Burke IT Administrator Department of Electrical Engineering and Computer Science University of Tennessee, Knoxville bbu...@eecs.utk.edu (865) 974-4694 ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/13/2011 9:03 PM, Bryan Burke wrote: My logs show successful update sources in the last line, but not when there is no update. Ok, well I did check the output of the grep before posting the number of lines on this list, and all log entries mentioning that IP were failures. So there's still *technically* some gray area, in that, if it happened to query that IP successfully, and there was no update, we'd never know, but I'm guessing that would reveal a similar outcome. There is no grey area. All connections are logged, both successful and unsuccessful. When DNS reports there is no update available, no connection is attempted and consequently there is no IP to log. From a well-connected host near Nashville TN USA: # tcping 88.198.67.125 80 88.198.67.125 port 80 closed. I get identical port 80 closed results from several hosts on various major USA ISPs. Logs going back a couple weeks show several failures each day and zero successful downloads from this host for us. While I certainly appreciate the donation of hardware and bandwidth by the owners of 88.198.67.125, a host that is consistently unavailable should be removed from the pool until it can be reliably accessed. -- Noel Jones ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/13/11 7:53 PM, Noel Jones wrote: On 9/13/2011 9:03 PM, Bryan Burke wrote: My logs show successful update sources in the last line, but not when there is no update. Ok, well I did check the output of the grep before posting the number of lines on this list, and all log entries mentioning that IP were failures. So there's still *technically* some gray area, in that, if it happened to query that IP successfully, and there was no update, we'd never know, but I'm guessing that would reveal a similar outcome. There is no grey area. All connections are logged, both successful and unsuccessful. When DNS reports there is no update available, no connection is attempted and consequently there is no IP to log. From a well-connected host near Nashville TN USA: # tcping 88.198.67.125 80 88.198.67.125 port 80 closed. I get identical port 80 closed results from several hosts on various major USA ISPs. I've just sent the URL to validator.wc3.org and got the same problem with this message: I got the following unexpected response when trying to retrieve http://88.198.67.125: 500 Can't connect to 88.198.67.125:80 (connect: Connection refused) I'm satisfied that site should be pulled from the list. If you have your own DNS server you can create your own round-robin authorative DNS server pointing to known to be reliable signature servers and which are located where ever they may be. It takes very little time to set one up. dp ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/13/11 8:05 PM, Dennis Peterson wrote: I've just sent the URL to validator.wc3.org and got the same problem with this message: My fat fingers intended to type http://validator.wc.org and not what they did type. dp ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/13/11 6:58 PM, Dennis Peterson denni...@inetnw.com wrote: On 9/13/11 3:15 PM, Bryan Burke wrote: At least concerning this issue, is there anything more to be done? Eliminate some unknowns - like maybe your DNS doesn't like big packets. Add this *temporarily* to your host table: 88.198.67.125 db.us.big.clamav.net And try again - and try with your browser, too. It should show you a web page indentifying the site you connected to and then after a short time you will be sent to clamav.net. Sounds like the server will be pulled, so you may not care, but since I went through the effort. Made changes to the hosts file. Ran dig $ db.us.clamav.net ; DiG 9.4.3-P3 db.us.clamav.net ;; global options: printcmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: NOERROR, id: 61401 ;; flags: qr rd ra; QUERY: 1, ANSWER: 16, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;db.us.clamav.net.INA ;; ANSWER SECTION: db.us.clamav.net.1190INCNAMEdb.us.big.clamav.net. db.us.big.clamav.net.50INA194.47.250.218 db.us.big.clamav.net.50INA194.186.47.19 db.us.big.clamav.net.50INA200.236.31.1 db.us.big.clamav.net.50INA204.109.62.22 db.us.big.clamav.net.50INA207.57.106.31 db.us.big.clamav.net.50INA208.72.56.53 db.us.big.clamav.net.50INA64.246.134.219 db.us.big.clamav.net.50INA65.19.179.67 db.us.big.clamav.net.50INA69.12.162.28 db.us.big.clamav.net.50INA69.163.100.14 db.us.big.clamav.net.50INA88.198.67.125 db.us.big.clamav.net.50INA150.214.142.197 db.us.big.clamav.net.50INA155.98.64.87 db.us.big.clamav.net.50INA168.143.19.95 db.us.big.clamav.net.50INA194.8.197.22 ;; Query time: 91 msec ;; SERVER: 10.0.1.1#53(10.0.1.1) ;; WHEN: Tue Sep 13 19:37:53 2011 ;; MSG SIZE rcvd: 298 Note that 88.198.67.125 is far down the list, so I immediately ran $ sudo /usr/local/clamXav/bin/freshclam --stdout --quiet --no-warnings --log=/usr/local/clamXav/share/clamav/freshclam.log With the following results: -- ClamAV update process started at Tue Sep 13 19:40:13 2011 main.cld is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven) connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Downloading daily-13610.cdiff [100%] Downloading daily-13611.cdiff [100%] daily.cld updated (version: 13611, sigs: 192595, f-level: 60, builder: guitar) bytecode.cld is up to date (version: 144, sigs: 41, f-level: 60, builder: edwin) Database updated (1038850 signatures) from db.US.clamav.net (IP: 69.163.100.14) So how could old 88 have possibly worked is way back to the top? Sending my browser to db.US.clamav.net gives me Safari can¹t open the page ³http://db.us.big.clamav.net/² because Safari can¹t connect to the server ³db.us.big.clamav.net². No matter how many times I try it. Was there anything else I need to try before restoring the hosts file? -Al- -- Al Varnell Mountain View, CA ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/13/11 8:07 PM, Dennis Peterson denni...@inetnw.com wrote: On 9/13/11 8:05 PM, Dennis Peterson wrote: I've just sent the URL to validator.wc3.org and got the same problem with this message: My fat fingers intended to type http://validator.wc.org and not what they did type. Or possibly http://validator.w3.org? -Al- -- Al Varnell Mountain View, CA ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/13/11 8:31 PM, Al Varnell wrote: Sounds like the server will be pulled, so you may not care, but since I went through the effort. Made changes to the hosts file. Ran dig $ db.us.clamav.net Does your dig use the host table? Mine does not. Same with nslookup. I can't imagine why they would, in fact. dp ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/13/11 8:34 PM, Al Varnell wrote: On 9/13/11 8:07 PM, Dennis Petersondenni...@inetnw.com wrote: On 9/13/11 8:05 PM, Dennis Peterson wrote: I've just sent the URL to validator.wc3.org and got the same problem with this message: My fat fingers intended to type http://validator.wc.org and not what they did type. Or possibly http://validator.w3.org? -Al- Thank you, Al - I knew the truth would out! The lesson learned is if you can avoid it, don't work 48 hour shifts and then try to think and type at the same time :) dp ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/13/11 8:34 PM, Dennis Peterson denni...@inetnw.com wrote: On 9/13/11 8:31 PM, Al Varnell wrote: Sounds like the server will be pulled, so you may not care, but since I went through the effort. Made changes to the hosts file. Ran dig $ db.us.clamav.net Does your dig use the host table? Mine does not. Same with nslookup. I can't imagine why they would, in fact. Apparently not. I re-launched Terminal, just in case that was necessary, but it still didn't make any difference. -Al- -- Al Varnell Mountain View, CA ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 09/13/2011 12:33 PM, Al Varnell wrote: On 9/13/11 10:18 AM, sys...@ra-schaal.desys...@ra-schaal.de wrote: Am 13.09.2011 18:01, schrieb Al Varnell: On Sep 13, 2011, at 8:15 AM, Dandantear...@gmail.com wrote: Yet more failure on 88.198.67.125, this morning. This one is a double. I was going to wait a few more days to mention this, but since you bring it up... I have seen this twice a day almost every day since 29 Aug. The only times I didn't see this was when the database was reported to be up-to-date. During that same period, I was _never_ able to successfully connect to it. This can't be just my bad luck. just your bad luck 2011/09/05 - 297638 connects 2011/09/06 - 265677 connects 2011/09/07 - 265228 connects 2011/09/08 - 210367 connects 2011/09/09 - 230462 connects 2011/09/10 - 142702 connects 2011/09/11 - 120486 connects 2011/09/12 - 207272 connects 2011/09/13 - 129521 connetcs until now - 1916 CET as mentioned a few days befor, YOU have a very slow connection to my system. I'm half a world away from you, so I'm not really surprised by that, but what difference should it make? just use another mirror instead of crying all the time about your bad setup. What are you talking about? I have no choice whatsoever on the mirror I connect to! -Al- Well here I have to take exception. You have every option to choose mirrors that suit your liking better. If the US servers are not meeting your needs, pick a different region. If the US round-robin are using mirrors half way around the world, then. there is no detraction to picking default mirrors that are half way around the world but choosing something other then US as the location. The fact that ClamXav HAS chosen to make it inconvenient for users to change update frequency or setting of db mirrors is NOT a clamav fault. The mechanism exists in freschclam but the port to OS X has chosen to ignore this very important feature. Would you like me to write a user interface application so OS X users can do this very simple preference setting? And don't get me started on some of the stupid approaches Apple has taken to a very simple to manage OS like FreeBSD. Although I choose express no opinion on the MACH kernel versus other kernels, the MACH kernel choice, is not issue that has detracted from the ability to easily set preferences. Apple has chosen to go the Microsoft route of our users are too stupid to be allowed to do their own customization and as such we OS X users have to suffer as we do with the choices made in Redmond. -- Jim Preston jimli...@commspeed.net ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 09/13/2011 01:16 PM, Chuck Swiger wrote: This being said, there is definitely a recurring issue with this particular mirror. Since Aug 22, I've seen: % grep Can't connect to port 80 of host database.clamav.net (IP: 88.198.67.125) /var/log/freshclam.log | wc -l 27 ...with zero successful connections to that IP. The connectivity failure is entirely reproducible by hand: % telnet 88.198.67.125 80 Trying 88.198.67.125... telnet: connect to address 88.198.67.125: Connection refused telnet: Unable to connect to remote host Well I wonder if it is a configuration issue on the web server of thus mirror. Others have reported that it responds to pings but will not accept connections on port 80. Maybe the config is unrealistically limiting connections.. I don't consider this to be a significant problem since other mirrors are up, but it's not a matter of bandwidth or connectivity on my side. As it happens, I'm testing from Cupertino, CA via Apple's 17.0.0.0/8 network, and from a Time-Warner cable link from NYC, NY on 24.103.0.0/16. Is Apple running an ISP on 17.0.0.0/8? If so, maybe my objection to Apple having a class A pubic subnet is unjustified. -- Jim Preston jimli...@commspeed.net ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On Sun, Sep 11, 2011 at 04:11:07PM -0400, Dan wrote: At 11:40 PM +0200 9/7/2011, Luca Gibelli wrote: Traffic is around 5TB/month on each mirror. Short of a paid service, which I doubt any of us want, few have such bandwidth available to donate. First of all, I think this whole thread is overreacting. I seriously doubt the mirror capacity is at maximum. Anyways, 5TB comes at 2MB/s average, which is not that much. I can do it with my $15 OVH/Kimsufi box and so do probably thousands of others. Clam needs to leverage the power of the Internet - as it is now, not yesterday. The simple, semi-linear propagate thru a few mirrors design has obviously reached a limit... 5 TB *per mirror* per month!!!??? Just to maintain a tiny 36 MB database? d'oh! It does sound a bit much for all the cdiffs etc, but maybe I'm underestimating the number of ClamAV users.. It may have worked just fine yesterday, but, seriously, just a model that's waiting to fall on its face as Clam becomes more popular. I don't think it can suddenly come _that_ much more popular, since it's already quite popular. So, I'm thinking that leaves two choices: 1) a cloud, a la Amazon S3. 2) p2p. Maybe, someday, when the well-cached cloud services are fully propagated *and* reliable world-wide, using a cloud in leiu of the traditional mirror set-up might be viable. But IMO that's years away and too expensive. There's nothing wrong with the current method. It's simple and cheap. You are underestimating the bandwidth available in the world. Either there really is no problem and ClamAV is just lazily fishing for more mirrors, or then they are just clueless and/or not having the substantial financial and engineering resources of a much larger organization (advertised in faq). Heck, even I could buy few boxes for mirrors, but I'm not going to do that as a private person since there are bazillion commercial entities that have or can get the bandwidth if needed, including Sourcefire itself. Right now, IMO, a p2p set-up would be the most viable. Continue to propagate via mirrors. *ADD* the torrent. Together, we clam users have many times the bandwidth needed! Is there a way to make freshclam grab and verify database files from a local directory? If there is, creating a torrent set-up would be fairly easy, even on an ad-hoc basis. I think it would be interesting to get a test going... WRT the reputation of p2p/torrents... There are quite a few legit uses for p2p. A number of open source products are even distributed via bittorrent. Yes, some ISPs are blocking the protocol -- but when shown that it's a legit use, they're usually willing to fix that. I like the idea of some 3rd party offering torrent service for the p2p-minded. What I don't want to see is freshclam bloated with some torrent libraries and stuff. You do realize that torrents actually need to have central servers for the .torrent files themselves? That's just the first step (freshclam would have already downloaded cdiffs at the same step). Then you actually need to have some trackers also, unless you are relying on DHT. Hopefully it's not the main database you end up downloading from some guys slow ADSL link.. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
At 9:22 AM +0300 9/12/2011, Henrik K wrote: On Sun, Sep 11, 2011 at 04:11:07PM -0400, Dan wrote: At 11:40 PM +0200 9/7/2011, Luca Gibelli wrote: Traffic is around 5TB/month on each mirror. Short of a paid service, which I doubt any of us want, few have such bandwidth available to donate. First of all, I think this whole thread is overreacting. I seriously doubt the mirror capacity is at maximum. Noone has suggested maximum. The issue is that the mirrors are so overloaded that it's often taking freshclam an excessive amount of time to do its thing, because of the time-outs / connection failures. No big deal if it's the update run in the background. But if it's on-demand update preceding a user-driven scan, it's making the user sit there, twiddling its thumbs, for up to a minute or two. Luca's response to the problem is that more mirror capacity is needed. Hence the discussion of alternatives... Anyways, 5TB comes at 2MB/s average, which is not that much. I can do it with my $15 OVH/Kimsufi box and so do probably thousands of others. Perhaps, where you live. Here, in the good'ole USofA, if I set up a server to feed 170 GB/day, my ISP would shut me down and bill me big. So, I'm thinking that leaves two choices: 1) a cloud, a la Amazon S3. 2) p2p. Maybe, someday, when the well-cached cloud services are fully propagated *and* reliable world-wide, using a cloud in leiu of the traditional mirror set-up might be viable. But IMO that's years away and too expensive. There's nothing wrong with the current method. It's simple and cheap. You are underestimating the bandwidth available in the world. I didn't say there's anything wrong with the current method. It's just overwhelmed, and I doubt that adding a mirror or two will fix it now or even in the long term. I'm looking to explore ways of supplementing the current infrastructure. You do realize that torrents actually need to have central servers for the .torrent files themselves? Are you saying that including a 30 KB file in the Clam distro is too heavy of a burden? That's just the first step (freshclam would have already downloaded cdiffs at the same step). Then you actually need to have some trackers also, unless you are relying on DHT. Hopefully it's not the main database you end up downloading from some guys slow ADSL link.. The point of a torrent is that noone provides all the data from one source. It's *distributed*. - Dan. -- - Psychoceramic Emeritus; South Jersey, USA, Earth. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
G.W. Haywood wrote: The ClamAV database mirrors appear to have a growing capacity problem. Torrents are intended to alleviate the problem, and it takes, oh, ten minutes to set one up. Scripts already exist which could be adapted fairly easily to use torrents instead of mirrors to download the data. The DNS tells us the filenames to ask for. Anybody can run a torrent, the torrent software can control the data rates used by clients, and a network of torrents is a much more challenging target for the Bad Guys than a few mirrors. So what's the problem? Maybe I just don't understand enough about how torrents actually work... but wouldn't you need to update the .torrent every time the virus database changed? I don't think the standard torrent protocol includes any support for something like that... -kgd ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/12/2011 11:05 AM, Dan wrote: At 9:22 AM +0300 9/12/2011, Henrik K wrote: On Sun, Sep 11, 2011 at 04:11:07PM -0400, Dan wrote: At 11:40 PM +0200 9/7/2011, Luca Gibelli wrote: Traffic is around 5TB/month on each mirror. Short of a paid service, which I doubt any of us want, few have such bandwidth available to donate. First of all, I think this whole thread is overreacting. I seriously doubt the mirror capacity is at maximum. Noone has suggested maximum. The issue is that the mirrors are so overloaded that it's often taking freshclam an excessive amount of time to do its thing, because of the time-outs / connection failures. No big deal if it's the update run in the background. But if it's on-demand update preceding a user-driven scan, it's making the user sit there, twiddling its thumbs, for up to a minute or two. Luca's response to the problem is that more mirror capacity is needed. Hence the discussion of alternatives... Anyways, 5TB comes at 2MB/s average, which is not that much. I can do it with my $15 OVH/Kimsufi box and so do probably thousands of others. Perhaps, where you live. Here, in the good'ole USofA, if I set up a server to feed 170 GB/day, my ISP would shut me down and bill me big. HERE HERE! My ISP is pretty cool about letting users do what they want. However, if I started moving 170GB / day they would definitely be chasing me down to have a chat. :-) When they start offering inexpensive 10Mbit links to the net, a mirror would be an option, but not right now. -- Sincerely, Nathan Gibbs Systems Administrator Christ Media http://www.cmpublishers.com signature.asc Description: OpenPGP digital signature ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On Mon, Sep 12, 2011 at 12:41:14PM -0400, Nathan Gibbs wrote: On 9/12/2011 11:05 AM, Dan wrote: At 9:22 AM +0300 9/12/2011, Henrik K wrote: On Sun, Sep 11, 2011 at 04:11:07PM -0400, Dan wrote: At 11:40 PM +0200 9/7/2011, Luca Gibelli wrote: Traffic is around 5TB/month on each mirror. Short of a paid service, which I doubt any of us want, few have such bandwidth available to donate. First of all, I think this whole thread is overreacting. I seriously doubt the mirror capacity is at maximum. Noone has suggested maximum. The issue is that the mirrors are so overloaded that it's often taking freshclam an excessive amount of time to do its thing, because of the time-outs / connection failures. No big deal if it's the update run in the background. But if it's on-demand update preceding a user-driven scan, it's making the user sit there, twiddling its thumbs, for up to a minute or two. Luca's response to the problem is that more mirror capacity is needed. Hence the discussion of alternatives... Anyways, 5TB comes at 2MB/s average, which is not that much. I can do it with my $15 OVH/Kimsufi box and so do probably thousands of others. Perhaps, where you live. Here, in the good'ole USofA, if I set up a server to feed 170 GB/day, my ISP would shut me down and bill me big. HERE HERE! My ISP is pretty cool about letting users do what they want. However, if I started moving 170GB / day they would definitely be chasing me down to have a chat. :-) When they start offering inexpensive 10Mbit links to the net, a mirror would be an option, but not right now. Guys, I'm not talking about some home or office ISP lines. I'm talking about rented dedicated servers that have huge bandwidth by contract. Why do you make pointless arguments? Depending on where you live or want the servers to be located, they can be cheap or amazingly cheap. And Dan, please familiarize yourself first on how torrents work. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On Sep 12, 2011, at 10:58 AM, Henrik K h...@hege.li wrote: I'm not talking about some home or office ISP lines. I'm talking about rented dedicated servers that have huge bandwidth by contract. Why do you make pointless arguments? Has anybody talked to Apple? Every box of Server software they sell comes with clavav, so they are already invested and have plenty of capacity world-wide. Sent from Janet's iPad -Al- -- Al Varnell ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
At 8:58 PM +0300 9/12/2011, Henrik K wrote: Guys, I'm not talking about some home or office ISP lines. I'm talking about rented dedicated servers that have huge bandwidth by contract. Why do you make pointless arguments? Excuse me? Pointless? Is that your way of disagreeing intelligently or just trying to shut the conversation down? In YOUR opinion individuals and even small businesses are incapable of contributing to Clam's strained infrastructure? So OUR suggestions and inquiries on this USER mailing list are ... pointless? And Dan, please familiarize yourself first on how torrents work. I know pretty much how they work. What's your point here? Is there some design issue that invalidates the idea of using a p2p/torrent type distribution method to supplement the mirrors? I just love having a design idea shot down with no discussion because it's POINTLESS. Or perhaps I've made the error here? Is there some heresy in asking my question yesterday: Is there a way to make freshclam grab and verify database files from a local directory? - Dan. -- - Psychoceramic Emeritus; South Jersey, USA, Earth. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 09/12/2011 10:54 PM, Dan wrote: Is there a way to make freshclam grab and verify database files from a local directory? Yes, but they don't work for fetching incremental updates from local dir (DatabaseCustomURL, PrivateMirror). What you could try is set DatabaseMirror to a local webserver, which fetches CDIFFs/CVDs from torrents on demand. FWIW fetching small cdiffs (1kb) via torrents is probably a bad idea as it'll take a lot more for you to find peers than to download from a mirror. Best regards, --Edwin ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 09/12/2011 12:20 PM, Al Varnell wrote: On Sep 12, 2011, at 10:58 AM, Henrik Kh...@hege.li wrote: I'm not talking about some home or office ISP lines. I'm talking about rented dedicated servers that have huge bandwidth by contract. Why do you make pointless arguments? Has anybody talked to Apple? Every box of Server software they sell comes with clavav, so they are already invested and have plenty of capacity world-wide. Sent from Janet's iPad -Al- And Apple (along with several other large corporations) has an over abundance of public IP addresses to assign to their own hosted servers 017/8 (16,777,216 IP Addresses). -- Jim Preston jimli...@commspeed.net ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On Sep 12, 2011, at 3:20 PM, Al Varnell wrote: On Sep 12, 2011, at 10:58 AM, Henrik K h...@hege.li wrote: I'm not talking about some home or office ISP lines. I'm talking about rented dedicated servers that have huge bandwidth by contract. Why do you make pointless arguments? Has anybody talked to Apple? Yes. ...and you know that's all I can say about it. -- Joel Esler OpenSource Community Manager Sourcefire ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On Mon, Sep 12, 2011 at 03:54:44PM -0400, Dan wrote: At 8:58 PM +0300 9/12/2011, Henrik K wrote: Guys, I'm not talking about some home or office ISP lines. I'm talking about rented dedicated servers that have huge bandwidth by contract. Why do you make pointless arguments? Excuse me? Pointless? Is that your way of disagreeing intelligently or just trying to shut the conversation down? In YOUR opinion individuals and even small businesses are incapable of contributing to Clam's strained infrastructure? So OUR suggestions and inquiries on this USER mailing list are ... pointless? I'm sorry but that's the fact. If mirrors need bandwidth, it's not going to work on some slow home connection. Why do you take it so personally? If you want to help, buy a server and host a mirror. And Dan, please familiarize yourself first on how torrents work. I know pretty much how they work. What's your point here? Is there some design issue that invalidates the idea of using a p2p/torrent type distribution method to supplement the mirrors? Obviously you didn't think how you are going to download all those cdiffs. You do realize that all of them need .torrent files also? It's pointless overhead. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On Mon, Sep 12, 2011 at 05:57:24PM -0400, Nathan Gibbs wrote: On 9/12/2011 1:58 PM, Henrik K wrote: Guys, I'm not talking about some home or office ISP lines. I'm talking about rented dedicated servers that have huge bandwidth by contract. OK, but what the rest of us are talking about is taking load off the global clamav mirror infrastructure. Particularly the US section. And I'm not?? But a da*n US server and host a mirror. Even as a individual if you like. Depending on where you live Because it is our section of the infrastructure that is having issues. Please read the thread title. Even I can buy some US servers if I want. There are lots of providers to choose from. or want the servers to be located, they can be cheap or amazingly cheap. I don't care where the servers are as long as I can get the current DBs. Rehash 1. The Clamav Project needs more capacity especially in the US zone. 2. Many of us have gone to a local mirror configuration to use as little of the capacity as possible. 3. The Clamav Project still needs more capacity. 4. Many of us would step up to the plate and provide this capacity if it were within our ability to do so. If you are an individual not able to put $15-$100 a month, then yes, it's not in your capability. 5. Barring that we are asking about torrent because we would step up to the plate and provide what is within our ability to provide. I could easily provide 20MB of transfer a month initially and maybe more. However 5TB / month is definitely out of the question. No one thinks any less of you for trying to help, on the contrary. But if you can't even get any facts straight etc, it's just messing up the thread. Let's not forget that ClamAV is backed by a commercial organization?? If they wanted US bandwidth badly, they can get it. If not by buying, then probably just by asking around or even on the web page? Why do you think it's not mentioned there. Probably very few users read this list. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
At 12:25 PM +0100 9/9/2011, G.W. Haywood wrote: On 8-9 Sep 2011 Luca Gibelli and I wrote: If anyone can provide a CVD mirror in US, please contact me directly. We definitely need more capacity in the db.us.clamav.net RR. I've asked this before and never had an answer, so I'll ask again. Is there a reason why torrents can't be used? On Thu, 08 Sep 2011 Jim Preston wrote: Torrents have a bad reputation from pirating software, illegal digital media distribution, distributing infections and malware and such. I am not sure I would trust torrents in a general way. Since the widespread adoption of strong encryption and digital certificates the Internet has been used securely to transfer verifiable and non-repudiable information. The published ClamAV databases are verifiably signed. Exactly. Either the pieces are already properly verified by freshclam before being added into the local database, or there is a massive security vulnerability that needs to be addressed immediately! ... spreading the work over many pipes ... That's the whole point, and that's why I keep banging on about it. :) At 11:40 PM +0200 9/7/2011, Luca Gibelli wrote: Traffic is around 5TB/month on each mirror. Short of a paid service, which I doubt any of us want, few have such bandwidth available to donate. Clam needs to leverage the power of the Internet - as it is now, not yesterday. The simple, semi-linear propagate thru a few mirrors design has obviously reached a limit... 5 TB *per mirror* per month!!!??? Just to maintain a tiny 36 MB database? d'oh! It may have worked just fine yesterday, but, seriously, just a model that's waiting to fall on its face as Clam becomes more popular. So, I'm thinking that leaves two choices: 1) a cloud, a la Amazon S3. 2) p2p. Maybe, someday, when the well-cached cloud services are fully propagated *and* reliable world-wide, using a cloud in leiu of the traditional mirror set-up might be viable. But IMO that's years away and too expensive. Right now, IMO, a p2p set-up would be the most viable. Continue to propagate via mirrors. *ADD* the torrent. Together, we clam users have many times the bandwidth needed! Is there a way to make freshclam grab and verify database files from a local directory? If there is, creating a torrent set-up would be fairly easy, even on an ad-hoc basis. I think it would be interesting to get a test going... WRT the reputation of p2p/torrents... There are quite a few legit uses for p2p. A number of open source products are even distributed via bittorrent. Yes, some ISPs are blocking the protocol -- but when shown that it's a legit use, they're usually willing to fix that. fwiw, - Dan. -- - Psychoceramic Emeritus; South Jersey, USA, Earth. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/9/11 3:07 PM, Nathan Gibbs wrote: Not everyone on this list works in your kind of shop. Our shop has a host whose main purpose in life is to torrent Debian ISO's. All the other person is asking, is why can't we have the capability to use torrents? This solution could take load off the global mirror infrastructure? I'm sure that many of us are already running local mirror configurations to do this, but obviously it isn't enough. The global infrastructure is still stressed. It takes next to no time to seed the signed signatures on a torrent system in your own home. Set it up and post a link. dp ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/8/11 11:54 AM, Jim Preston jimli...@commspeed.net wrote: The last issue is what is the default country code for ClamXav? I have not installed or configured ClamXav for a long time. I know that the default source build for clamav is #DatabaseMirror db.XY.clamav.net and by incorrect default configuration just uses DatabaseMirror database.clamav.net. Again, this is not for this list to discuss specifically, but if ClamXav users are not selecting the proper database round robin for their locale, may be the package maintainers should force an interactive selection during the installation / configuration. You are correct that the default is db.XY.clamav.net. In working with users I have observed that the network is pretty good about picking the correct list to use based on what I understand is a complicated geographic analysis of the client's IP address. I have not observed any users being routed to mirrors great distances away by using the default and as far as US users are concerned I find their results almost always identical to mine at any given time. Because changing the CC involves non-trivial actions by a Mac user (most don't even know where to find the Terminal app, let alone use it) and it must be repeated each time the engine is reinstalled, I only recommend it to those who have a need to update their database more often than every two hours, since that's a clamav.net requirement or insist that they need to make the change due to other network issues. I have personally written a short script to accomplish this, but don't always remember to use it. I once thought about recommending to Mark that he make it a Preference option, but never felt that it was truly necessary. He's got his hands full just keeping up with engine and OS changes, and there are several other improvements that I feel are more important right now. -Al- -- Al Varnell Mountain View, CA ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
Hi there, On 8-9 Sep 2011 Luca Gibelli and I wrote: If anyone can provide a CVD mirror in US, please contact me directly. We definitely need more capacity in the db.us.clamav.net RR. I've asked this before and never had an answer, so I'll ask again. Is there a reason why torrents can't be used? like twitter, torrent is not a requirement (we are planning to extend this rule to everything that starts with a 't'). Still no answer then? :) On Thu, 08 Sep 2011 Jim Preston wrote: Torrents have a bad reputation from pirating software, illegal digital media distribution, distributing infections and malware and such. I am not sure I would trust torrents in a general way. Last I heard, the Internet had a similar reputation, but I guess you still use it. Of course in parts of the UK they call it 'tinternet' so Luca might say it's not a requirement. :) Since the widespread adoption of strong encryption and digital certificates the Internet has been used securely to transfer verifiable and non-repudiable information. The published ClamAV databases are verifiably signed. ... spreading the work over many pipes ... That's the whole point, and that's why I keep banging on about it. :) The ClamAV database mirrors appear to have a growing capacity problem. Torrents are intended to alleviate the problem, and it takes, oh, ten minutes to set one up. Scripts already exist which could be adapted fairly easily to use torrents instead of mirrors to download the data. The DNS tells us the filenames to ask for. Anybody can run a torrent, the torrent software can control the data rates used by clients, and a network of torrents is a much more challenging target for the Bad Guys than a few mirrors. So what's the problem? -- 73, Ged. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/9/11 4:25 AM, G.W. Haywood wrote: So what's the problem? I guess I'd like to see what your Checkpoint firewall rules in your DC look like and read your presentation to your security team justifying connecting your system to unknown systems using a distribution method most better known for software and music pirating for the purpose of uploading AV signatures from your AV vendor, and that this is needed because the vendor actually doesn't have enough bandwidth to do the job right without this torrent method. dp ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 09/09/2011 04:25 AM, G.W. Haywood wrote: Last I heard, the Internet had a similar reputation, but I guess you still use it. Of course in parts of the UK they call it 'tinternet' so Luca might say it's not a requirement. :) Since the widespread adoption of strong encryption and digital certificates the Internet has been used securely to transfer verifiable and non-repudiable information. The published ClamAV databases are verifiably signed. I am on the Internet? OMG, where is that plug? gots ta pull it ;-) -- Jim Preston jimli...@commspeed.net ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/8/2011 11:41 AM, Luca Gibelli wrote: Hello G.W., If anyone can provide a CVD mirror in US, please contact me directly. We definitely need more capacity in the db.us.clamav.net RR. I've asked this before and never had an answer, so I'll ask again. Is there a reason why torrents can't be used? like twitter, torrent is not a requirement (we are planning to extend this rule to everything that starts with a 't'). They must have extended it from everything that starts with s. When I asked about https support in freshclam, I was told SSL wasn't required. Security isn't required either. CLAM BAKE! :-) -- Sincerely, Nathan Gibbs Systems Administrator Christ Media http://www.cmpublishers.com signature.asc Description: OpenPGP digital signature ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/9/2011 10:57 AM, Dennis Peterson wrote: On 9/9/11 4:25 AM, G.W. Haywood wrote: So what's the problem? I guess I'd like to see what your Checkpoint firewall rules in your DC look like and read your presentation to your security team justifying connecting your system to unknown systems using a distribution method most better known for software and music pirating for the purpose of uploading AV signatures from your AV vendor, and that this is needed because the vendor actually doesn't have enough bandwidth to do the job right without this torrent method. dp Not everyone on this list works in your kind of shop. Our shop has a host whose main purpose in life is to torrent Debian ISO's. All the other person is asking, is why can't we have the capability to use torrents? This solution could take load off the global mirror infrastructure? I'm sure that many of us are already running local mirror configurations to do this, but obviously it isn't enough. The global infrastructure is still stressed. -- Sincerely, Nathan Gibbs Systems Administrator Christ Media http://www.cmpublishers.com signature.asc Description: OpenPGP digital signature ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
Before I respond I think I should describe my role in this community. I provide uncompensated tech support on the ClamXav forum, so I attempt to keep my system configured as close to defaults as possible, since that's what most users do. On Sep 7, 2011, at 9:49 PM, Jim Preston jimli...@commspeed.net wrote: I do see your concern if as your log shows you are only checking twice a day Missing an update extends the stale db quite a bit. Three questions; Do you have both of these lines in your freshclam.conf? DatabaseMirror db.us.clamav.net DatabaseMirror database.clamav.net What is your max retry before failure? Default is 3 Yes to all three, but most of the users I assist do not update the country code. How many updates are you making? Default in freshclam says it is every 2 hours or 12/day but . you do not seem to be making that many based on your log posting. Just two, but that actually more than meets my needs. Most ClamXav users update as needed or once a day, at most. I do help a few sysadmins with critical server responsibilities who require 12 or more updates a day, however. One of them reported 30 failures involving this server in a 24-hour period involving multiple machines. Right / wrong / or indifferent, I am running freshclam once an hour off the hour. I am doing this via a cron task and select the execution minute. To help reduce load, I do not run it on the hour but have selected sometime after the hour. ClamXav just switch from using cron to lauchd, which has been the preferred method for event scheduling in Mac OS X for some time. So right now I have one event scheduled by launchd and a legacy event using cron to troubleshoot various issues concerning both. Appreciate the input, but the issue I am trying to solve here is simply to improve the reliability of the CVD mirror network for US users. Sent from Janet's iPad -Al- -- Al Varnell ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
Hi there, On Thu, 8 Sep 2011 Luca Gibelli wrote: ... If anyone can provide a CVD mirror in US, please contact me directly. We definitely need more capacity in the db.us.clamav.net RR. ... I've asked this before and never had an answer, so I'll ask again. Is there a reason why torrents can't be used? -- 73, Ged. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
Hello G.W., If anyone can provide a CVD mirror in US, please contact me directly. We definitely need more capacity in the db.us.clamav.net RR. I've asked this before and never had an answer, so I'll ask again. Is there a reason why torrents can't be used? like twitter, torrent is not a requirement (we are planning to extend this rule to everything that starts with a 't'). Best regards -- Luca Gibelli (luca _at_ clamav.net) ClamAV, a GPL anti-virus toolkit [Tel] +39 0187 1851862 [Fax] +39 0187 1852252 [IM] nervous/jabber.linux.it PGP key id 5EFC5582 @ any key-server || http://www.clamav.net/gpg/luca.gpg ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
-Original Message- From: clamav-users-boun...@lists.clamav.net [mailto:clamav-users- boun...@lists.clamav.net] On Behalf Of Luca Gibelli Hello G.W., If anyone can provide a CVD mirror in US, please contact me directly. We definitely need more capacity in the db.us.clamav.net RR. I've asked this before and never had an answer, so I'll ask again. Is there a reason why torrents can't be used? like twitter, torrent is not a requirement (we are planning to extend this rule to everything that starts with a 't'). No typing required? ;-) ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 09/07/2011 11:14 PM, Al Varnell wrote: Before I respond I think I should describe my role in this community. I provide uncompensated tech support on the ClamXav forum, so I attempt to keep my system configured as close to defaults as possible, since that's what most users do. On Sep 7, 2011, at 9:49 PM, Jim Prestonjimli...@commspeed.net wrote: I do see your concern if as your log shows you are only checking twice a day Missing an update extends the stale db quite a bit. Three questions; Do you have both of these lines in your freshclam.conf? DatabaseMirror db.us.clamav.net DatabaseMirror database.clamav.net What is your max retry before failure? Default is 3 Yes to all three, but most of the users I assist do not update the country code. How many updates are you making? Default in freshclam says it is every 2 hours or 12/day but . you do not seem to be making that many based on your log posting. Just two, but that actually more than meets my needs. Most ClamXav users update as needed or once a day, at most. I do help a few sysadmins with critical server responsibilities who require 12 or more updates a day, however. One of them reported 30 failures involving this server in a 24-hour period involving multiple machines. Right / wrong / or indifferent, I am running freshclam once an hour off the hour. I am doing this via a cron task and select the execution minute. To help reduce load, I do not run it on the hour but have selected sometime after the hour. ClamXav just switch from using cron to lauchd, which has been the preferred method for event scheduling in Mac OS X for some time. So right now I have one event scheduled by launchd and a legacy event using cron to troubleshoot various issues concerning both. Appreciate the input, but the issue I am trying to solve here is simply to improve the reliability of the CVD mirror network for US users. Sent from Janet's iPad -Al- Thanks Al, The first line it the most telling. I was not aware of this and the service you are providing, and well my bad, just made an assumption you were just bitching for the sake of bitching without making simple modifications to your system. I have read many of your posts and should have realized there was a deeper reason. I am not going to go through the previous posts to be sure, but .. I do not remember (at least in this last server issue) that you were supporting CLamXav. I too support and have several Apple OS X systems and am aware of the launchd preference over cron but that is a separate bitch of mine to be take up on an Apple forum ;-) The last issue is what is the default country code for ClamXav? I have not installed or configured ClamXav for a long time. I know that the default source build for clamav is #DatabaseMirror db.XY.clamav.net and by incorrect default configuration just uses DatabaseMirror database.clamav.net. Again, this is not for this list to discuss specifically, but if ClamXav users are not selecting the proper database round robin for their locale, may be the package maintainers should force an interactive selection during the installation / configuration. Thanks for allowing me to put in my 2 cents and and no I am not trying to start a flame war nor looking for a response from anyone but Al. -- Jim Preston jimli...@commspeed.net ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 09/08/2011 03:14 AM, G.W. Haywood wrote: I've asked this before and never had an answer, so I'll ask again. Is there a reason why torrents can't be used? -- Against my head's better judgment, I am going to respond here =-O Torrents have a bad reputation from pirating software, illegal digital media distribution, distributing infections and malware and such. I am not sure I would trust torrents in a general way. However, It does sound like a very intriguing idea for distributing the db to my own server farms, spreading the work over many pipes and systems . I was always very intrigued by the jigdo distribution for Debian and used it when I was experimenting with Debian as my main *nix platform. Thanks, Jim -- Jim Preston jimli...@commspeed.net ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
Hello Al, error. Since that time each of two updates on 2, 3, 4, 5, 6 Sep have started with that same server and erred with the following: connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) That status page has shown some issues with that server each day, but nothing like what I am seeing. the admin of akxnet.de has limited the amount of concurrent connections on the mirror. Depending on traffic, you may get a connection refused error, but it's nothing to worry about. freshclam will just try to connect to another mirror in the RR. If anyone can provide a CVD mirror in US, please contact me directly. We definitely need more capacity in the db.us.clamav.net RR. Regards, -- Luca Gibelli (luca _at_ clamav.net) ClamAV, a GPL anti-virus toolkit [Tel] +39 0187 1851862 [Fax] +39 0187 1852252 [IM] nervous/jabber.linux.it PGP key id 5EFC5582 @ any key-server || http://www.clamav.net/gpg/luca.gpg ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
Am 07.09.2011 05:11, schrieb Al Varnell: According to my mirrors.dat file the last time I successfully connected to the US Mirror at akxnet.de (IP: 88.198.67.125) (obviously located in DE) was on 29 Aug, but when I check my log I see that even that was actually an error. Since that time each of two updates on 2, 3, 4, 5, 6 Sep have started with that same server and erred with the following: connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) That status page has shown some issues with that server each day, but nothing like what I am seeing. I can ping the server, but can not find it with my browser. When I enter clamav.akxnet.de I am currently taken to a different IP address, I believe (46.4.61.241). The 88.198.67.125 address comes back to mx00.akxnet.de -Al- You can use 46.4.61.241 and 88.198.67.125 - both systems are always in sync. Due to heavy traffic (up to 5 GB in july only for the mirror) i limited the access. 2 connects/ip at the same time and 500 current connections. The error should not allways apear. If so, could you please mail me your ip so i can have a look at the firewall. regards Florian ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/7/2011 7:13 AM, Luca Gibelli wrote: If anyone can provide a CVD mirror in US, please contact me directly. We definitely need more capacity in the db.us.clamav.net RR. If I had the bandwidth, I would. When I have the bandwidth, I intend to. -- Sincerely, Nathan Gibbs Systems Administrator Christ Media http://www.cmpublishers.com signature.asc Description: OpenPGP digital signature ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On Wed, 7 Sep 2011, Luca Gibelli wrote: If anyone can provide a CVD mirror in US, please contact me directly. We definitely need more capacity in the db.us.clamav.net RR. What sort of bandwidth do the mirrors use, as in what would be a typical burst or peak load - 5mbit/sec, 10mbit/sec, etc. == Chris Candreva -- ch...@westnet.com -- (914) 948-3162 WestNet Internet Services of Westchester http://www.westnet.com/ ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On Wed, Sep 07, 2011 at 01:13:37PM +0200, Luca Gibelli wrote: Hello Al, error. Since that time each of two updates on 2, 3, 4, 5, 6 Sep have started with that same server and erred with the following: connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) That status page has shown some issues with that server each day, but nothing like what I am seeing. the admin of akxnet.de has limited the amount of concurrent connections on the mirror. Depending on traffic, you may get a connection refused error, but it's nothing to worry about. freshclam will just try to connect to another mirror in the RR. If anyone can provide a CVD mirror in US, please contact me directly. We definitely need more capacity in the db.us.clamav.net RR. Is Sourceforge so cheap that it can't get few $20 vps for mirrors? ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/7/11 4:21 AM, sys...@ra-schaal.de sys...@ra-schaal.de wrote: Am 07.09.2011 05:11, schrieb Al Varnell: According to my mirrors.dat file the last time I successfully connected to the US Mirror at akxnet.de (IP: 88.198.67.125) (obviously located in DE) was on 29 Aug, but when I check my log I see that even that was actually an error. Since that time each of two updates on 2, 3, 4, 5, 6 Sep have started with that same server and erred with the following: connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) That status page has shown some issues with that server each day, but nothing like what I am seeing. I can ping the server, but can not find it with my browser. When I enter clamav.akxnet.de I am currently taken to a different IP address, I believe (46.4.61.241). The 88.198.67.125 address comes back to mx00.akxnet.de -Al- You can use 46.4.61.241 and 88.198.67.125 - both systems are always in sync. I don't really get a choice, I must use whatever is handed off to me. I have never seen the 46 IP on our list. Due to heavy traffic (up to 5 GB in july only for the mirror) i limited the access. 2 connects/ip at the same time and 500 current connections. I did a port scan last evening and 80 never came available: Port Scanning host: 88.198.67.125 Open TCP Port: 21 ftp Open TCP Port: 25 smtp Open TCP Port: 53 domain Open TCP Port: 110pop3 Open TCP Port: 143imap Open TCP Port: 221fln-spx Also, there are two other ClamXav users in different parts of the country that have been reporting this issue. The error should not allways apear. If so, could you please mail me your ip so i can have a look at the firewall. Currently 71.198.46.64 but it's dynamically assigned by the ISP, so could change. regards Florian -Al- ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
Am 07.09.2011 19:59, schrieb Al Varnell: You can use 46.4.61.241 and 88.198.67.125 - both systems are always in sync. I don't really get a choice, I must use whatever is handed off to me. I have never seen the 46 IP on our list. You can use clamav.akxnet.de: nslookup clamav.akxnet.de Server: 127.0.0.1 Address:127.0.0.1#53 Name: clamav.akxnet.de Address: 88.198.67.125 Name: clamav.akxnet.de Address: 46.4.61.241 Maybe Luca can the second IP to db.us.big.clamav.net. Due to heavy traffic (up to 5 GB in july only for the mirror) i limited the access. 2 connects/ip at the same time and 500 current connections. I did a port scan last evening and 80 never came available: Port Scanning host: 88.198.67.125 Open TCP Port: 21 ftp Open TCP Port: 25 smtp Open TCP Port: 53 domain Open TCP Port: 110pop3 Open TCP Port: 143imap Open TCP Port: 221fln-spx Due to some problems with our backbone yesterday, this may be a reason. wc -l database.clamav.net/200.log 202168 database.clamav.net/200.log The error should not allways apear. If so, could you please mail me your ip so i can have a look at the firewall. Currently 71.198.46.64 but it's dynamically assigned by the ISP, so could change. I´ve seen your IP within the last three days only two times. BTW: ping 71.198.46.64 PING 71.198.46.64 (71.198.46.64) 56(84) bytes of data. 64 bytes from 71.198.46.64: icmp_seq=1 ttl=236 time=189 ms 64 bytes from 71.198.46.64: icmp_seq=2 ttl=236 time=189 ms That´s not really fast. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/7/2011 1:59 PM, Al Varnell wrote: I did a port scan last evening and 80 never came available: Port Scanning host: 88.198.67.125 Open TCP Port: 21 ftp Open TCP Port: 25 smtp Open TCP Port: 53 domain Open TCP Port: 110pop3 Open TCP Port: 143imap Open TCP Port: 221fln-spx Bad Sysadmin, no coffee! -- Sincerely, Nathan Gibbs Systems Administrator Christ Media http://www.cmpublishers.com signature.asc Description: OpenPGP digital signature ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 9/7/11 11:19 AM, sys...@ra-schaal.de sys...@ra-schaal.de wrote: Am 07.09.2011 19:59, schrieb Al Varnell: You can use 46.4.61.241 and 88.198.67.125 - both systems are always in sync. I don't really get a choice, I must use whatever is handed off to me. I have never seen the 46 IP on our list. You can use clamav.akxnet.de: nslookup clamav.akxnet.de Server: 127.0.0.1 Address:127.0.0.1#53 Name: clamav.akxnet.de Address: 88.198.67.125 Name: clamav.akxnet.de Address: 46.4.61.241 I don't see that as a work around unless... Maybe Luca can the second IP to db.us.big.clamav.net. Could Luca use the url instead of the IP address in db.us.big.clamav.net? Due to heavy traffic (up to 5 GB in july only for the mirror) i limited the access. 2 connects/ip at the same time and 500 current connections. I did a port scan last evening and 80 never came available: Port Scanning host: 88.198.67.125 Open TCP Port: 21 ftp Open TCP Port: 25 smtp Open TCP Port: 53 domain Open TCP Port: 110pop3 Open TCP Port: 143imap Open TCP Port: 221fln-spx Due to some problems with our backbone yesterday, this may be a reason. wc -l database.clamav.net/200.log 202168 database.clamav.net/200.log I've started a new port scan. The error should not allways apear. If so, could you please mail me your ip so i can have a look at the firewall. Currently 71.198.46.64 but it's dynamically assigned by the ISP, so could change. I´ve seen your IP within the last three days only two times. Here's my freshclam log for that period with six attempts: -- ClamAV update process started at Sun Sep 4 15:50:19 2011 main.cld is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven) connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Downloading daily-13548.cdiff [100%] Downloading daily-13549.cdiff [100%] daily.cld updated (version: 13549, sigs: 186393, f-level: 60, builder: guitar) bytecode.cld is up to date (version: 144, sigs: 41, f-level: 60, builder: edwin) Database updated (1032648 signatures) from db.US.clamav.net (IP: 194.186.47.19) -- ClamAV update process started at Mon Sep 5 00:16:32 2011 main.cld is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven) connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Downloading daily-13550.cdiff [100%] Downloading daily-13551.cdiff [100%] daily.cld updated (version: 13551, sigs: 186402, f-level: 60, builder: guitar) bytecode.cld is up to date (version: 144, sigs: 41, f-level: 60, builder: edwin) Database updated (1032657 signatures) from db.US.clamav.net (IP: 65.19.179.67) WARNING: Clamd was NOT notified: Can't connect to clamd through /tmp/clamd.socket -- ClamAV update process started at Mon Sep 5 07:45:02 2011 main.cld is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven) connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Downloading daily-13552.cdiff [100%] Downloading daily-13553.cdiff [100%] Downloading daily-13554.cdiff [100%] Downloading daily-13555.cdiff [100%] daily.cld updated (version: 13555, sigs: 186543, f-level: 60, builder: ccordes) bytecode.cld is up to date (version: 144, sigs: 41, f-level: 60, builder: edwin) Database updated (1032798 signatures) from db.US.clamav.net (IP: 168.143.19.95) -- ClamAV update process started at Mon Sep 5 15:45:48 2011 main.cld is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven) connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Downloading daily-13556.cdiff [100%] Downloading daily-13557.cdiff [100%] Downloading daily-13558.cdiff [100%] Downloading daily-13559.cdiff [100%] daily.cld updated (version: 13559, sigs: 187320, f-level: 60, builder: ccordes) bytecode.cld is up to date (version: 144, sigs: 41, f-level: 60, builder: edwin) Database updated (1033575 signatures) from db.US.clamav.net (IP: 69.163.100.14) -- ClamAV update process started at Tue Sep 6 07:45:01 2011 main.cld is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven) connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Downloading daily-13560.cdiff [100%] Downloading daily-13561.cdiff [100%] Downloading daily-13562.cdiff [100%] Downloading daily-13563.cdiff [100%] daily.cld updated (version: 13563, sigs:
Re: [clamav-users] Yet Another US Mirror Issue
Hello Christopher, If anyone can provide a CVD mirror in US, please contact me directly. We definitely need more capacity in the db.us.clamav.net RR. What sort of bandwidth do the mirrors use, as in what would be a typical burst or peak load - 5mbit/sec, 10mbit/sec, etc. You can throttle the bandwidth to whatever you can afford, we usually require a minimum of 10Mbit/s though. Traffic is around 5TB/month on each mirror. Best regards -- Luca Gibelli (luca _at_ clamav.net) ClamAV, a GPL anti-virus toolkit [Tel] +39 0187 1851862 [Fax] +39 0187 1852252 [IM] nervous/jabber.linux.it PGP key id 5EFC5582 @ any key-server || http://www.clamav.net/gpg/luca.gpg ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On Sep 7, 2011, Luca Gibelli l...@clamav.net wrote: Hello Al, error. Since that time each of two updates on 2, 3, 4, 5, 6 Sep have started with that same server and erred with the following: connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) That status page has shown some issues with that server each day, but nothing like what I am seeing. the admin of akxnet.de has limited the amount of concurrent connections on the mirror. Depending on traffic, you may get a connection refused error, but it's nothing to worry about. freshclam will just try to connect to another mirror in the RR. Yes, I am aware of that, in fact it did so in every case and since I do updates on a scheduled basis, it's no real impact. The problem comes when a user has need for a spontaneous scan of a file and requests a definition update before the scan. At this point he must wait an additional 30 seconds which, over the course of a day, results in unacceptable (to him) non-productive time. In his view, such a connection error should result in a failure being logged and eventually blacklisting that server. I suggested to him that he file a bug report to thst effect, but the problem at that time was resolved, so elected not to bother. Sent from Janet's iPad -Al- -- Al Varnell ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [clamav-users] Yet Another US Mirror Issue
On 09/07/2011 12:36 PM, Al Varnell wrote: Here's my freshclam log for that period with six attempts: -- ClamAV update process started at Sun Sep 4 15:50:19 2011 main.cld is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven) connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Downloading daily-13548.cdiff [100%] Downloading daily-13549.cdiff [100%] daily.cld updated (version: 13549, sigs: 186393, f-level: 60, builder: guitar) bytecode.cld is up to date (version: 144, sigs: 41, f-level: 60, builder: edwin) Database updated (1032648 signatures) from db.US.clamav.net (IP: 194.186.47.19) -- ClamAV update process started at Mon Sep 5 00:16:32 2011 main.cld is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven) connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Downloading daily-13550.cdiff [100%] Downloading daily-13551.cdiff [100%] daily.cld updated (version: 13551, sigs: 186402, f-level: 60, builder: guitar) bytecode.cld is up to date (version: 144, sigs: 41, f-level: 60, builder: edwin) Database updated (1032657 signatures) from db.US.clamav.net (IP: 65.19.179.67) WARNING: Clamd was NOT notified: Can't connect to clamd through /tmp/clamd.socket -- ClamAV update process started at Mon Sep 5 07:45:02 2011 main.cld is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven) connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Downloading daily-13552.cdiff [100%] Downloading daily-13553.cdiff [100%] Downloading daily-13554.cdiff [100%] Downloading daily-13555.cdiff [100%] daily.cld updated (version: 13555, sigs: 186543, f-level: 60, builder: ccordes) bytecode.cld is up to date (version: 144, sigs: 41, f-level: 60, builder: edwin) Database updated (1032798 signatures) from db.US.clamav.net (IP: 168.143.19.95) -- ClamAV update process started at Mon Sep 5 15:45:48 2011 main.cld is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven) connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Downloading daily-13556.cdiff [100%] Downloading daily-13557.cdiff [100%] Downloading daily-13558.cdiff [100%] Downloading daily-13559.cdiff [100%] daily.cld updated (version: 13559, sigs: 187320, f-level: 60, builder: ccordes) bytecode.cld is up to date (version: 144, sigs: 41, f-level: 60, builder: edwin) Database updated (1033575 signatures) from db.US.clamav.net (IP: 69.163.100.14) -- ClamAV update process started at Tue Sep 6 07:45:01 2011 main.cld is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven) connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Downloading daily-13560.cdiff [100%] Downloading daily-13561.cdiff [100%] Downloading daily-13562.cdiff [100%] Downloading daily-13563.cdiff [100%] daily.cld updated (version: 13563, sigs: 187384, f-level: 60, builder: jesler) bytecode.cld is up to date (version: 144, sigs: 41, f-level: 60, builder: edwin) Database updated (1033639 signatures) from db.US.clamav.net (IP: 69.163.100.14) -- ClamAV update process started at Tue Sep 6 15:46:56 2011 main.cld is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven) connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) Downloading daily-13564.cdiff [100%] Downloading daily-13565.cdiff [100%] Downloading daily-13566.cdiff [100%] Downloading daily-13567.cdiff [100%] Downloading daily-13568.cdiff [100%] Downloading daily-13569.cdiff [100%] Downloading daily-13570.cdiff [100%] daily.cld updated (version: 13570, sigs: 187667, f-level: 60, builder: jesler) bytecode.cld is up to date (version: 144, sigs: 41, f-level: 60, builder: edwin) Database updated (1033922 signatures) from db.US.clamav.net (IP: 194.8.197.22) Hi Al, I do see your concern if as your log shows you are only checking twice a day Missing an update extends the stale db quite a bit. Three questions; Do you have both of these lines in your freshclam.conf? DatabaseMirror db.us.clamav.net DatabaseMirror database.clamav.net What is your max retry before failure? Default is 3 How many updates are you making? Default in freshclam says it is every 2 hours or 12/day but . you do not seem to be making that many based on your log posting. Right / wrong / or indifferent, I am running freshclam once an hour off the hour. I am doing this via a cron
[clamav-users] Yet Another US Mirror Issue
According to my mirrors.dat file the last time I successfully connected to the US Mirror at akxnet.de (IP: 88.198.67.125) (obviously located in DE) was on 29 Aug, but when I check my log I see that even that was actually an error. Since that time each of two updates on 2, 3, 4, 5, 6 Sep have started with that same server and erred with the following: connect_error: getsockopt(SO_ERROR): fd=4 error=61: Connection refused Can't connect to port 80 of host db.US.clamav.net (IP: 88.198.67.125) That status page has shown some issues with that server each day, but nothing like what I am seeing. I can ping the server, but can not find it with my browser. When I enter clamav.akxnet.de I am currently taken to a different IP address, I believe (46.4.61.241). The 88.198.67.125 address comes back to mx00.akxnet.de -Al- -- Al Varnell Mountain View, CA ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
