I think these reports don't tell you what you think they mean. In fact they're
pretty much meaningless. The two different servers have different versions of
the signature. That is perfectly normal - there is simply zero chance and it is
naive to think they will always be fully synced in the same second of time of
day. You can infer nothing when this occurs.
In any event these signature serial numbers are associated with the DNS txt
record. The designed process is entirely serial - freshclam knows your installed
signature file serial number, it knows the DNS txt record, and it requests
updates from any of the signature servers if the local version is different from
the DNS txt record. It will try all the mirrors until success or the list of
mirrors is exhausted. Other things that mess with the fully synchronized state
is that DNS caching, TTL, local system clock differences, and policies of
various name service admins to ignore authoritative TTL suggestions.
The database.clamav.net dns is a round robin of 5 different servers and you
cannot predict what you will receive. In fact in the best case the list be
reordered each time you request the A record. And the chances of two different
clients getting the same A record is very low.
Your own local resolver looks in its own cache to see if it has expired. The TTL
record for the TXT record is 1800 seconds. If you use the dig command retrieve
the TXT record you can watch the TTL count down:
dig txt current.cvd.clamav.net |grep TXT
To eliminate this as a problem source you can always use host table entries
rather than dns for your tests. The round robin records ensure reliability for
the client and crude load balancing for the server farm.
So worst case is the record you see can be 1800 seconds behind an updated TXT
record. Obviously polling the current.cvd.clamav.net server directly will return
an uncached record at the expense of recursing queries (use the IP instead of
the hostname to avoid this).
Because these variables exist, freshclam is somewhat fault tolerant and will
retry 3 times per mirror (default and is configurable), and if a mirror is in a
failed state freshclam will map it out of the servers to try next time
(mirrors.dat). The other variable is some of the sync process is demand-driven.
In very busy systems (which these are) stale files should not exist very long.
Your request just might be a trigger to refresh a stale file, and the next
person to hit that server will retrieve the updated file and your system will
move to another mirror. This scenario presumes files are pulled to the mirrors,
not pushed.
I do believe your angst over not having complete system synchronization is
unwarranted as there are too many uncontrollable variables and it's really not
critical if the first mirror doesn't respond.
Finally - the current cloudflare process is pretty solid - it is a vast
improvement over the historical mirror collaboration
On 11/26/18 4:19 PM, Paul Kosinski wrote:
I believe that the delays we have been observing are due to some
problem with the Boston Cloudflare servers, or, perhaps, Comcast has a
"transparent" caching proxy which is causing us trouble.
I recently installed the same build and configuration of ClamAV 0.100.2
on our Web server, a virtual machine hosted in NYC. It runs the same
extra code (curl etc.) to check the cvd version number that we have
locally. Since Friday, there have been no delays there, although there
have been several significant delays locally. They check at exactly
the same time as each other (i.e., via NTP synced cron jobs).
I also am now running, at each location, simple curls to read the first
few bytes of the cvd files (to get the version number), *and* to log
all the headers sent and received. These are also run at exactly the
same time (as each other) via cron.
The headers show that our local system uses the 'BOS' Cloudflare server,
while the remote one uses the 'IAD' server:
CF-RAY: 47fd0b7af79dae32-BOS
CF-RAY: 47fd0b8064d9c1b8-IAD
Interestingly, these two cron jobs sometimes show that the BOS server
is out of date relative to the IAD server. For example, the following
curls show that one cvd file served by the BOS server is one version
behind that served by the IAD server at the *same* time. The files'
"Last-modified" lines are of particular interest. The BOS server says
the file was last modified on Mon, 26 Nov 2018 at 06:19:22 GMT, while
the IAD server says the file was last modified on Mon, 26 Nov 2018 at
14:15:24 GMT.
In particular, the BOS "Date:" header says it's already about 14 mins
*later* than the IAD "Last-modified:" timestamp indicates. In other
words, the file delivered by the BOS server is, at time of *delivery*,
already about 14 minutes out of date.
_______________________________________________
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml
