I've been doing some testing ove the last couple of days with 3.02 and found it's scores are way lower on all test emails than 2.64. (anywhere upto 33% lower in limited tests).
I've managed to get most of my 2.64 rules etc over (along with bayes), but I'm nervous about switching given the amount of spam 2.64 IS catching vs 3.02 MIGHT miss.
Alot of the defaults rules have reduced scores (esp when running bayes+net combination) and I don't want to give my users the spam.
I've seen several other people that complain/note this issue as well so it seems I'm not alone on this.
I shall be sticking to 2.64 for the forsee-able future as 3.02 gives me no advantage and quite a high likelihood of more spam dropping through the system!
Question on this test:
Did you use SA 2.64 stock, or with add-ons? Most importantly, did you use the SpamCopURI plugin with 2.64.
One thing to keep in mind is that by adding rules, particularly SpamCopURI, to 2.64 without adjusting any other scores, you've effectively increased the average score of a spam message by a great deal.
This is what happens whenever you add spam rules that haven't been run through the GA and/or perceptron.
You get a higher average score, lower FN rate, higher FP rate. It's the way of the universe unless the added rules have a true 0% FP rate. I know of very few rules with a true 0FP rate. Clearly little or nothing from SpamAssassin's standard rules, SARE, or any other add-on source would get 0 FP's in a 100K ham/100k spam email test unless it matched no spam either.
SA generally tunes it's scores in a VERY conservative manner, with the assumption that FPs are 100 times worse than FNs. The default SpamCopURI scores clearly don't reflect this mentality, as the default scores are much more likely to cause 1 FP than 100 FNs.
The default scores for Mail::SpamCopURI were set up using the SARE general guideline of looking at the S/O of the rule and scoring it. Unfortunately, this doesn't take into account overlap.
If SC fires, WS nearly ALWAYS fires too. On my system, less than 5% of SPAMCOP_URI_RBL matches did not also match WS_URI_RBL.
Less than 10% of SC matches did not also match OB.
But when you combine the two, Less than 0.65% of SPAMCOP_URI_RBL hits matched neither WS nor OB.
The combined score of spamcop with either of the other two rules under Mail::SpamCopURI is 5.1, This effectively makes the spamcop URI list implicitly trusted for 99.35% of it's hits. The spamcop URI list has an impressively low FP rate, but it's not zero.
Be careful when comparing SA versions that you're comparing apples to apples. Don't compare SA 3.0.2 to an unbalanced version of 2.64 which has an unrealistically high score bias, and FP rate, compared to the stock system.
