Re: [clamav-users] local server takes time to update clamav db

2020-12-13 Thread Dave Warren via clamav-users
Okay, so then it seems like 1) ClamAV’s origin server periodically serves an old version of a file after the DNS TXT record is updated, or 2) Cloudflare returns a cached resource from the wrong URL, or 3) Someone is making a request to new ?version URLs before the DNS TXT record is updated (and

Re: [clamav-users] local server takes time to update clamav db

2020-12-13 Thread Paul Kosinski via clamav-users
I did a quick grep on the the source code (and compiled output too) of ClamAV 0.103.0, and I couldn't find any instance of 'CF-Cache-Status'. Should freshclam (or somebody) be checking this HTTP header line that Cloudflare returns? The 'STALE' and 'UPDATING' values sound like they might be

Re: [clamav-users] local server takes time to update clamav db

2020-12-13 Thread Joel Esler (jesler) via clamav-users
Both of those things are done as well. Sent from my  iPhone > On Dec 13, 2020, at 19:24, Dave Warren via clamav-users > wrote: > > On 2020-12-11 08:51, Paul Kosinski via clamav-users wrote: >> "The whole CVD filename is not versioned (always "daily.cvd") which is >> why the CloudFlare

Re: [clamav-users] local server takes time to update clamav db

2020-12-13 Thread Dave Warren via clamav-users
On 2020-12-11 08:51, Paul Kosinski via clamav-users wrote: "The whole CVD filename is not versioned (always "daily.cvd") which is why the CloudFlare caching issue may result in serving the previous version." HTML filenames for Web pages are not versioned either. Does this mean that CDNs like

Re: [clamav-users] local server takes time to update clamav db

2020-12-11 Thread Joel Esler (jesler) via clamav-users
this "wait a while and try >> again" thing is transparent to you. Disable the `Verbose` option in >> freshclam.conf and don't worry about it. >> >> -Micah >> >>> -Original Message- >>> From: clamav-users On Behalf Of >>

Re: [clamav-users] local server takes time to update clamav db

2020-12-11 Thread Paul Kosinski via clamav-users
to you. Disable the `Verbose` option in freshclam.conf and > don't worry about it. > > -Micah > > > -Original Message----- > > From: clamav-users On Behalf Of > > G.W. Haywood via clamav-users > > Sent: Thursday, December 10, 2020 9:21 AM > > To: Jo

Re: [clamav-users] local server takes time to update clamav db

2020-12-11 Thread Paul Kosinski via clamav-users
Does ClamAV (Talos?) check *all* the Cloudflare anycast servers? I thought it could only check those "near" to ClamAV POPs. On Thu, 10 Dec 2020 18:00:15 + "Joel Esler (jesler)" wrote: > > On Dec 10, 2020, at 11:58 AM, Paul Kosinski via clamav-users > > wrote: > > > > I would imagine

Re: [clamav-users] local server takes time to update clamav db

2020-12-10 Thread Micah Snyder (micasnyd) via clamav-users
mav-users On Behalf Of > G.W. Haywood via clamav-users > Sent: Thursday, December 10, 2020 9:21 AM > To: Joel Esler (jesler) via clamav-users > Cc: G.W. Haywood > Subject: Re: [clamav-users] local server takes time to update clamav db > > Hi there, > > On Thu, 10 Dec 2

Re: [clamav-users] local server takes time to update clamav db

2020-12-10 Thread Joel Esler (jesler) via clamav-users
> On Dec 10, 2020, at 12:21 PM, G.W. Haywood via clamav-users > wrote: > > But the real fix must be in the hands of Cloudflare, or perhaps those > of Cloudflare's customers (making more fuss about something which, at > first sight, could very easily be remedied). While I agree, I am sure

Re: [clamav-users] local server takes time to update clamav db

2020-12-10 Thread Joel Esler (jesler) via clamav-users
> On Dec 10, 2020, at 11:58 AM, Paul Kosinski via clamav-users > wrote: > > I would imagine that Cloudflare has a means of fetching a specific file > from any of their own mirror servers (via its unique, non-anycast, IP > address) to check its operation. If ClamAV DB files could be requested

Re: [clamav-users] local server takes time to update clamav db

2020-12-10 Thread G.W. Haywood via clamav-users
Hi there, On Thu, 10 Dec 2020, Joel Esler (jesler) via clamav-users wrote: I think the way to fix this is, freshclam, if it receives an "I'm behind" error from the PoP, to do a sleep for awhile and then try again. ... Maybe the workaround is simpler than that. The document at

Re: [clamav-users] local server takes time to update clamav db

2020-12-10 Thread Paul Kosinski via clamav-users
With regard to "sleep for awhile". I remember that Cloudflare's BOS server on occasion remained behind the latest CVD version (according to the DNS TXT record) for more than one hour! Might the following be possible instead? I would imagine that Cloudflare has a means of fetching a specific

Re: [clamav-users] local server takes time to update clamav db

2020-12-10 Thread Andrew C Aitchison via clamav-users
On Thu, 10 Dec 2020, Joel Esler (jesler) via clamav-users wrote: So, there are occasions where one PoP from Cloudflare is behind and hasn't yet fetched the file from the other PoP or from our mirror directly. This might be the case that you're the first one that's asked for it from your PoP

Re: [clamav-users] local server takes time to update clamav db

2020-12-10 Thread Joel Esler (jesler) via clamav-users
> On Dec 10, 2020, at 9:07 AM, Andrew C Aitchison > wrote: > > On Thu, 10 Dec 2020, Joel Esler (jesler) via clamav-users wrote: > >>> >>> >> >> By “unable to resolve” Micah means: “There’s nothing more we can do >> to solve the problem”. >> >> When we generate a new cvd/cld,

Re: [clamav-users] local server takes time to update clamav db

2020-12-10 Thread Andrew C Aitchison via clamav-users
Date: Thu, 10 Dec 2020 14:07:08 + (GMT) From: Andrew C Aitchison To: clamav-users@lists.clamav.net Cc: "Joel Esler (jesler)" Subject: Re: [clamav-users] local server takes time to update clamav db On Thu, 10 Dec 2020, Joel Esler (jesler) via clamav-users wrote: On Dec 10, 202

Re: [clamav-users] local server takes time to update clamav db

2020-12-10 Thread Joel Esler (jesler) via clamav-users
> On Dec 10, 2020, at 6:06 AM, G.W. Haywood via clamav-users > wrote: > > Hi there, > > On Wed, 9 Dec 2020, Micah Snyder (micasnyd) via clamav-users wrote: > >> This message: >> “*The daily.cvd database downloaded from https://database.clamav.net >> is one version older than advertised in

Re: [clamav-users] local server takes time to update clamav db

2020-12-10 Thread G.W. Haywood via clamav-users
Hi there, On Wed, 9 Dec 2020, Micah Snyder (micasnyd) via clamav-users wrote: This message: “*The daily.cvd database downloaded from https://database.clamav.net is one version older than advertised in the DNS TXT record.” is a verbose-level message from freshclam but is not an error to

Re: [clamav-users] local server takes time to update clamav db

2020-12-09 Thread Micah Snyder (micasnyd) via clamav-users
] local server takes time to update clamav db Hello, I'm serving cvd files from a local server, when I run freshclam on my server it takes some runes until the daily.cvd is updated even though the remote version was updated a while ago. - the clamav version I'm using is 0.102.4-r1

Re: [clamav-users] local server takes time to update clamav db

2020-12-09 Thread Paul Kosinski via clamav-users
"This is one of the IPs which I was expecting to see. I wouldn't expect any problems with it, our ClamAV server updated from it at 1818 GMT last night." Unfortunately, given the way Cloudflare works, the IP address (e.g., 104.16.218.84) isn't the whole story. A particular Anycast IP address such

Re: [clamav-users] local server takes time to update clamav db

2020-12-09 Thread Gal Cohen
The same problem happens to me now when trying to update from 26012 to 26013 version I was trying to wget directly from each IP (104.16.218.84, 104.16.219.8) and still I'm getting 26012 version from them. maybe this relates to the fact `scriptUpdates no`? even though the server should be configure

Re: [clamav-users] local server takes time to update clamav db

2020-12-09 Thread G.W. Haywood via clamav-users
Hi there, On Wed, 9 Dec 2020, Gal Cohen wrote: 5. here are the full logs of the latest update failure (26011 -> 26012),freshclam run takes 19 sec Tue Dec 8 22:00:02 2020 -> ClamAV update process started at Tue Dec 8 22:00:02 2020 ... Tue Dec 8 22:00:02 2020 ->

Re: [clamav-users] local server takes time to update clamav db

2020-12-09 Thread Gal Cohen
Hi all, 1. the daily.cvd file I referring to is on the local server which acts as the private local mirror (not referring to clients at all) 2. The freshclam is running in a docker , his image came from `alpine:3.12` and clamav was install with that command: `apk add --no-cache

Re: [clamav-users] local server takes time to update clamav db

2020-12-08 Thread G.W. Haywood via clamav-users
Hi there, On Tue, 8 Dec 2020, Gal Cohen wrote: I'm serving cvd files from a local server, when I run freshclam on my server it takes some runes until the daily.cvd is updated even though the remote version was updated a while ago. Please clarify that the daily.cvd file which you are talking

[clamav-users] local server takes time to update clamav db

2020-12-08 Thread Gal Cohen
Hello, I'm serving cvd files from a local server, when I run freshclam on my server it takes some runes until the daily.cvd is updated even though the remote version was updated a while ago. - the clamav version I'm using is 0.102.4-r1 - freshclam.conf I'm using is: DatabaseDirectory /data