On 01/10/2018 01:25 PM, Jens Schleusener wrote:
On Wed, 10 Jan 2018, Dave Jones wrote:
On 01/10/2018 08:48 AM, Kevin A. McGrail wrote:
Can you turn on debugging and perhaps add it to retry again? I am
trying to figure out if it is one server with an issue.
We have added a number of new sa-update mirrors recently. Check the
MIRRORED.BY file and do ping/traceroutes AND wget/curls to each
server. There could be a local routing problem getting to one of them
from your location/ISP.
https://svn.apache.org/viewvc/spamassassin/site/updates/MIRRORED.BY?revision=1819744&view=markup
Dave
I am the maintainer of one of the new sa-update mirrors
(sa-update.fossies.org).
Just an observation (although I am not very familiar with the complete
update mechanismn):
For e.g. today between
10/Jan/2018:09:34:29 +0100
and
10/Jan/2018:09:40:04 +0100
I saw in the web logs of the mirror 76 GET requests to /1820725.tar.gz
with a 404 ("Not Found") response code (only an that time interval).
The file 1820725.tar.gz has on the mirror server the last modification
date "Jan 10 09:31" and the rsync logs shows that the file
1820725.tar.gz was fetched at
Jan 10 09:40:11 CET 2018
So some client hosts have probably the information that 1820725.tar.gz is
the freshest update file before the mentioned mirror server has rsynced
that file.
Similar effects I found in the days before with roughly 80 "404 (Not
Found)" requests against roughly 61000 "200 (Ok)" requests.
Can it be possible that the failed SHA1 verification is caused by that
effect?
If yes, is the mirror frequency too low (on sa-update.fossies.org
currently 10 minutes) or is the information about the current update
file too early available to the clients?
But maybe I have misinterpreted the situation.
Regards
Jens
I think you are spot on. The DNS updates used to have a delay to give
the mirrors time to update. The DNS TTL for the TXT records is
currently 1 hour. I realize that some DNS caches that don't have the
TXT record already cached are going to update quickly a few seconds
after the TXT is updated with the new ruleset information.
It does look like there is a few minutes time when DNS has updated
before all mirrors are sync'd so I will add a 10 minute delay to the DNS
updates to give the mirrors time to pull the latest rulesets.
Dave
