Re: sa-update.fossies.org (144.76.163.196) is longer down
On Mon, 18 Jul 2022, Jens Schleusener wrote: Hi, unfortunately the server sa-update.fossies.org (144.76.163.196) has after a reboot (Mo Jul 18 15:36:32 CEST) strong hardware problems. Since there is no foreseeable end to the server outage, I propose to remove the server from the mirror list. There are again problems (probably originally kernel based but now with internal ones). May be that is the end of the server fossies.org so please remove sa-update.fossies.org from the mirror list. Sorry Jens
sa-update.fossies.org (144.76.163.196) is longer down
Hi, unfortunately the server sa-update.fossies.org (144.76.163.196) has after a reboot (Mo Jul 18 15:36:32 CEST) strong hardware problems. Since there is no foreseeable end to the server outage, I propose to remove the server from the mirror list. Sorry Jens Schleusener
Re: Some interesting (?) observations on a mirror server (sa-update.fossies.org)
On Sat, 22 Sep 2018, Dave Jones wrote: On 9/20/18 2:50 PM, Fossies Administrator wrote: Hi, incidentally I looked some weeks ago on the web server access log file of the SpamAssassin rules update files mirror sa-update.fossies.org and found surprisingly that at noon (midday) the log file has a size much more than the roughly expected half of a complete daily log. Just for curiosity I plotted the number of the GET requests for update files (tarballs) per hour and saw an interesting characteristics with a great peak between 6 and 7 a.m. (GMT+2). Ok, the main reason is probably the publication time (mostly between 5 and 6 a.m. GMT+2) with a delay til the user's sa-update scripts are running. But the structure of the curves with the some curious (?) mimima is a little bit "surprisingly" to me but it is constant and reproducible. A simple example text plot for a single day is attached (more accurate plots are available under the URL given below). But more interesting and "irritating" was the fact that I found in the main update time often (at least 100-1000) entries with the HTTP status 404 ("Not Found"). That motivated me to write a primitive script to analyze the reason by monitoring the update status resp. update times of the new published rules update files. First I checked the local web log files assuming that a 404 request to an update file means that an external client had the information about a new file that the local mirror sa-update.fossies.org has not yet available resp. not yet fetched (via rsync). Additionally I checked the local DNS server (of the server provider) and the DNS servers I found responsible for the domain spamassassin.org ns2.pccc.com. ns2.ena.com. c.auth-ns.sonic.net. b.auth-ns.sonic.net. a.auth-ns.sonic.net. via the command dig @ 3.3.3.updates.spamassassin.org txt +short The plots and an extract of the script output you can find under https://fossies.org/~schleusener/sa-update.mirror_analysis/ User: sa PW: update The main reason for the 404 errors seems to be that the mirroring script is started as cronjob on sa-update.fossies.org only every 10 minutes. Probably better would be to check the original nameservers (the local nameserver answers according the TTL only with a freshness delay of max. one hour) and start only a rsync job if the response shows that a new file is available. If all mirror servers would use update frequencies not smaller than 10 minutes an idea may be also to set/change the DNS TXT entry only 10 minutes after the release (availability) of a new update file. Additionally I found that the synchronization of the above DNS servers seems delayed by some minutes. The "best" DNS server seems to be "ns2.ena.com" since it always as first one provides the new versions. Maybe this behaviour is a little bit related to the current thread with the subject "repeated sa-update problems" on the users list. Regards Jens Very interesting and useful information. Thank you Jens. I have put a 20 minute sleep in the script before the DNS updates happen to give the mirrors time to update before sa-update starts looking for the new ruleset. I run ns2.ena.com and it's updating quickly because it's receiving the DNS NOTIFY from the hidden master and performing a zone transfer immediately. Now this will happen after a 20 minute delay. All other DNS servers must be ignoring the NOTIFY and updating at the normal REFRESH interval in the SOA record which is 7200 so they will average out to be 1 hour delay behind the hidden master. [djones@djones5 trunk]$ svn diff Index: build/mkupdates/mkupdate-with-scores === --- build/mkupdates/mkupdate-with-scores(revision 1841667) +++ build/mkupdates/mkupdate-with-scores(working copy) @@ -282,6 +282,8 @@ if [ $AUTOUPDATESDISABLED -eq 1 -a $REVERT_REVISION -eq 0 ]; then echo "DNS updating disabled (auto update publishing disabled), skipping DNS reload" else +# Wait 20 minutes for the mirrors to update via rsync +sleep 1200 # Newer versions >= 3.4.1 of SpamAssassin are CNAME'd to 3.3.3 /usr/local/bin/updateDNS.sh 3.3.3.updates TXT $REVISION RC=$? [djones@djones5 trunk]$ svn commit -m "Added DNS update delay to give time for the mirrors to update via rsync before sa-update will start looking for the new rule sets." Sendingbuild/mkupdates/mkupdate-with-scores Transmitting file data .done Committing transaction... Committed revision 1841668. Dave After more than two weeks of observation I just want to confirm that your measure succeeds: Since September 23, there was not a single 404 error for an update file found on the mirror server sa-update.fossies.org. Jens
Re: Fwd: [Bug 7331] channel: SHA1 verification failed, channel failed
On Wed, 10 Jan 2018, Dave Jones wrote: On 01/10/2018 08:48 AM, Kevin A. McGrail wrote: Can you turn on debugging and perhaps add it to retry again? I am trying to figure out if it is one server with an issue. We have added a number of new sa-update mirrors recently. Check the MIRRORED.BY file and do ping/traceroutes AND wget/curls to each server. There could be a local routing problem getting to one of them from your location/ISP. https://svn.apache.org/viewvc/spamassassin/site/updates/MIRRORED.BY?revision=1819744&view=markup Dave I am the maintainer of one of the new sa-update mirrors (sa-update.fossies.org). Just an observation (although I am not very familiar with the complete update mechanismn): For e.g. today between 10/Jan/2018:09:34:29 +0100 and 10/Jan/2018:09:40:04 +0100 I saw in the web logs of the mirror 76 GET requests to /1820725.tar.gz with a 404 ("Not Found") response code (only an that time interval). The file 1820725.tar.gz has on the mirror server the last modification date "Jan 10 09:31" and the rsync logs shows that the file 1820725.tar.gz was fetched at Jan 10 09:40:11 CET 2018 So some client hosts have probably the information that 1820725.tar.gz is the freshest update file before the mentioned mirror server has rsynced that file. Similar effects I found in the days before with roughly 80 "404 (Not Found)" requests against roughly 61000 "200 (Ok)" requests. Can it be possible that the failed SHA1 verification is caused by that effect? If yes, is the mirror frequency too low (on sa-update.fossies.org currently 10 minutes) or is the information about the current update file too early available to the clients? But maybe I have misinterpreted the situation. Regards Jens On 1/10/2018 9:25 AM, Dale Blount wrote: I get them randomly starting a few months back. My cronjob is set for 4:40am Eastern. Normally it won't fail two days in a row. My cron script looks like this: /usr/bin/vendor_perl/sa-update --gpgkey 6C6191E3 --channel updates.spamassassin.org RET=$? if [ "$RET" -eq 0 ]; then /usr/bin/vendor_perl/sa-compile && systemctl restart spamassassin fi On 01/10/2018 09:09 AM, Kevin A. McGrail wrote: Anyone having issues with Sha1 failures on their machines on sa-updates? Anyone familiar with sa-update.cron so we can try and get more data on this bug below? Forwarded Message Subject: [Bug 7331] channel: SHA1 verification failed, channel failed Date: Tue, 09 Jan 2018 15:05:37 + From: bugzilla-dae...@issues.apache.org To: kmcgr...@apache.org https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7331 --- Comment #5 from Jonathan Kamens --- (In reply to Kevin A. McGrail from comment #4) > Please add more logs and if you can, try manually downloading the files. I'm getting the error from sa-update.cron, so (a) I'm not around when it happens in the middle of the night to retry it immediately, and (b) I have no idea where, if anywhere, the logs from sa-update.cron are captured. If you can advise me how to configure or modify the cron job so that it captures logs, I will be glad to follow your advice to collect additional information. > My big question is does a subsequent run fix the issue? Is there a specific > mirror that might be having the issue? When I get the error overnight and then rerun the update during the day when I notice it, it usually works the second time.