On 10/19/2017 03:49 AM, Merijn van den Kroonenberg wrote:
On 10/18/2017 10:10 AM, Merijn van den Kroonenberg wrote:
A new file has been created, 72_scores_temporary_fix.cf (107 lines),
that is the delta of the last known good 72_scores.cf from March and
the
latest incomplete 72_scores.cf (56 lines):
So the combination of 72_scores_temporary_fix.cf and the latest
72_scores.cf result in the same rules + scores as the 72_scores.cf from
march?
Correct.
Or are some scores different?
The ~56 scores in 72_scores.cf will change slightly each night based on
masscheck processing. The "last known good" scores from March for the
~107 that are missing from 72_scores.cf have been put in the
72_scores_temporary_fix.cf to prevent low overall scoring like what
happened in mid June.
Good point. Thanks for catching that. Another option is to scrap the
72_scores_temporary_fix.cf and make 72_scores.cf a static file in the
nightly masscheck script and ignore the generated version. I am open to
suggestions.
What is the intention of this temporary fix of the rule updates? Will it
allow score changes by the score generation system? If so, what will
prevent unexpected/unintended scores to be generated because the score
generation is broken? And if not, what is gained by activating the rule
updates again, will manual rule updates be pushed?
Yes. The scores that are making it to the 72_scores.cf appear to be
correct. The issue is causing 2/3rds of the scores to be completely
missing taking the default score of 1.0 to throw off scoring
significantly.
Right, but what I remembered from looking at this earlier, the generated
72_scores.cf (like the ones generated in june) seem to 'leave out'
different rules each time. Most of those ~56 are the same rules, but they
are not exactly. So each day some other rules are 'missing' and get the
default 1.0 score?
I compared the new 72_scores.cf against a few from june and what i
remembered seems to be correct, different rules disappear each day.
How are you countering this behaviour with a fixed
72_scores_temporary_fix.cf file? Or are you re-generating the
72_scores_temporary_fix.cf anew each day by comparing to the march 72
score file?
I could do this too with a little more effort in the nightly masscheck
scripts.
John Hardin said in an earlier mail: "The problem is 72_scores has
explicit bad scores in it.". Is this true? If so you override some rules
which do get generated in the ~56 set?
I am not sure what John meant as an action item from that statement.
Can anyone provide a good/safe 72_scores.cf file to put in the default
SA ruleset?
Bottom line is we need to get the nightly masscheck scripts and DNS
updates going again for sa-update to start working again.
Thank you for your help, Merijn. I really need another set of eyes
looking at the problem. I have been looking at this issue too long.
BTW, this issue doesn't impact my SA instances. I have so much local
customization and MTA optimizations that I didn't even notice the
problem back in June when we had a couple of weeks of low scoring. I am
trying to help the entire SA community with improvements to the default
ruleset.
We need to sa-updates going again. None of the updates of the past few
months here have gone out to the Internet:
http://svn.apache.org/viewvc/spamassassin/trunk/rules/?sortby=date
There are some other new rule updates that could be coming soon (i.e.
new LASHBACK RBL testing with very low scores) which add to the
importance to get updates enabled again while I continue to troubleshoot
the incomplete 72_scores.cf issue.
--
David Jones
http://svn.apache.org/viewvc/spamassassin/trunk/rulesrc/scores/72_scores_temporary_fix.cf
The latest ruleset 1812374.tar.gz with the 72_scores_temporary_fix.cf
was installed on my production mail filters and my spamtrap for
masschecking and the scoring is normal the past couple of hours. Last
time there was a sharp drop in scoring that should have been quickly
obvious.
I need some volunteers to manually install the latest ruleset from any
SA update mirror and check their scoring so we can enable automated
updates via DNS soon:
cd /tmp
wget http://sa-update.ena.com/1812374.tar.gz
wget http://sa-update.ena.com/1812374.tar.gz.sha1
wget http://sa-update.ena.com/1812374.tar.gz.asc
sa-update -v --install 1812374.tar.gz
(restart your spamd, amavisd, mimedefang, MailScanner, etc.)
Please run the commands above and provide some feedback so we can
enable
automatic sa-updates again soon.
--
David Jones
--
David Jones
--
David Jones
