Re: best way to whitelist this list?

[Ian Zimmerman]

On 2015-09-19 20:12 +0200, A. Schulze wrote:

> today I was notified by ezmlm that my MTA rejected messages to
> me. Messages to this list where classified as spam by .. spamassassin.

All of today's messages here scored around -7.5 for me, with no special
handling.

-- 
Please *no* private copies of mailing list or newsgroup messages.
Rule 420: All persons more than eight miles high to leave the court.

Re: best way to whitelist this list?

[Reindl Harald]

Am 19.09.2015 um 20:12 schrieb A. Schulze:

today I was notified by ezmlm that my MTA rejected messages to me.
Messages to this list where classified as spam by .. spamassassin.
OK, no surprise some messages look spammy.

As usual: there is one solution that is smart, fast and obvious. But
sometimes this one is also wrong.
So I ask the list: (how) do you whitelist this list?


whitelist_auth *@spamassassin.apache.org



signature.asc
Description: OpenPGP digital signature

best way to whitelist this list?

[A. Schulze]

Hello,

today I was notified by ezmlm that my MTA rejected messages to me. Messages to 
this list where classified as spam by .. spamassassin.
OK, no surprise some messages look spammy.

As usual: there is one solution that is smart, fast and obvious. But sometimes 
this one is also wrong.
So I ask the list: (how) do you whitelist this list?

Thanks,
Andreas

Re: best way to whitelist this list?

[Benny Pedersen]

A. Schulze skrev den 2015-09-19 20:12:


So I ask the list: (how) do you whitelist this list?


X-Spam-Status: No, score=-2.5 required=5.0 
tests=AWL,DKIM_SIGNED,DKIM_VALID,

DKIM_VALID_AU,RCVD_IN_DNSWL_HI,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,
	SPF_HELO_NONE shortcircuit=no autolearn=ham autolearn_force=no 
version=3.4.1


seem to me its not needed :=)

but sat whitelist_from_dkim your own dkim signature if needed

do more lines of that if more is needed, eg dkim signed, but content is 
spamming, the, most importange part possible is to eg disable autolearn, 
with imho happens if std rules are in use


missing that point will learn spam as ham in bayes

why is this maillist here missing spf helo pass ?, hmm

Help with RegEx Rule

[AK]

Hi all.

I'm getting hit with lots of JUNK mail that has multiple lines with just 
a '.' on several lines [0].  Most of the JUNK email has at least 5 and 
at most 10 lines (so far) with just this '.' character somewhere in the 
middle of the message.


I've copied the message source to RegexBuddy [1] and have been able to 
come up with a regex that matches what I want using the Perl 5.20 engine:


(^\.\n){5,}

However, adding this rule to /etc/spamassassin/local.cf doesn't hit at 
all when I run it against my test message as follows:


= Start Rule Block =
rawbody __MANY_PERIODS_1 ALL =~ /(^\.\n){5,}/
meta MANY_PERIODS __MANY_PERIODS_1
score MANY_PERIODS 2.0
describe MANY_PERIODS JUNK mail with several lines that contain single dot
= End Rule Block =

= Begin Test Command =
spamassassin -L -t test.msg
= End Test Command =


Please help me understand what I'm doing wrong as this is my first 
attempt at creating a rule.  Previously I've just copied and pasted what 
I've found here in the forums, but this time I'm trying to do it myself 
but failing.



Regards,
ak.


[0] - http://pastebin.com/NwrwCKjZ
[1] - http://www.regexbuddy.com/create.html

Re: Help with RegEx Rule

[Benny Pedersen]

On September 19, 2015 4:52:30 PM AK  wrote:


= Start Rule Block =
rawbody __MANY_PERIODS_1 ALL =~ /(^\.\n){5,}/


remove ALL =~, my own rawbody rules dont have it

Re: Help with RegEx Rule

[Bill Cole]

On 19 Sep 2015, at 10:51, AK wrote:


Hi all.

I'm getting hit with lots of JUNK mail that has multiple lines with 
just a '.' on several lines [0].  Most of the JUNK email has at least 
5 and at most 10 lines (so far) with just this '.' character somewhere 
in the middle of the message.


I've copied the message source to RegexBuddy [1] and have been able to 
come up with a regex that matches what I want using the Perl 5.20 
engine:


(^\.\n){5,}

However, adding this rule to /etc/spamassassin/local.cf doesn't hit at 
all when I run it against my test message as follows:


= Start Rule Block =
rawbody __MANY_PERIODS_1 ALL =~ /(^\.\n){5,}/
meta MANY_PERIODS __MANY_PERIODS_1
score MANY_PERIODS 2.0
describe MANY_PERIODS JUNK mail with several lines that contain single 
dot

= End Rule Block =

= Begin Test Command =
spamassassin -L -t test.msg
= End Test Command =


Please help me understand what I'm doing wrong as this is my first 
attempt at creating a rule.  Previously I've just copied and pasted 
what I've found here in the forums, but this time I'm trying to do it 
myself but failing.


There are  multiple issues...

0. I have no basis to criticize RegexBuddy specifically but as a general 
principle, that class of tool is usually more of a hindrance than an aid 
for understanding what you're doing with regular expressions. If you're 
using SA for anything more than your personal email (i.e. if you're 
managing a mail system that uses SA) you really need to learn regular 
expressions well enough to write them yourself.


1. As Benny noted, the '=~' isn't used in rawbody or body rules. It is 
the Perl regex-match operator that is used in header rules between the 
name of the header to be checked and the regex to be matched. I think 
'spamassassin --lint' would have identified that as bogus, and it is 
always good practice to run that after adding new rules.


2. The 'meta' rule structure is pointlessly complex (but see (4) below.)

3. To match across multiple lines, you need the 'm' modifier.

4. You might find it more flexible to make the base rule match '^\.$' 
with a tflags setting of 'multiple' and set one or more meta rules for 5 
or more hits OR just make the base rule a normal rule with a score and 
let the multiple hits add up.

Re: best way to whitelist this list?

[Bill Cole]

On 19 Sep 2015, at 14:12, A. Schulze wrote:


Hello,

today I was notified by ezmlm that my MTA rejected messages to me. 
Messages to this list where classified as spam by .. spamassassin.

OK, no surprise some messages look spammy.

As usual: there is one solution that is smart, fast and obvious. But 
sometimes this one is also wrong.

So I ask the list: (how) do you whitelist this list?


A layered defense in depth requires matching layered whitelisting...

All of my list-sub addresses (which match a pattern) are exempted from 
some relatively error-prone (~0.01% FP) tactics ahead of content 
filtering and all content filtering except SA. Inside SA, I use 
whitelist_to to further exempt the target. However: I have 
USER_IN_WHITELIST_TO reduced to -3 because these addresses are *more* 
exposed than a normal address and shouldn't get the standard -6. To 
further protect actual list mail (in contrast with off-list replies and 
spam from harvesters,) I use whitelist_from_spf where possible 
(including all apache.org lists,) falling back to whitelist_from_rcvd 
for other lists. The few lists I've subscribed to over the years where 
neither of those have been usable have all shut down anyway.

Re: Help with RegEx Rule

[Adam Major]

Hello

If you using compiled rules you probably should use:

sa-compile command and restart (if use :) sa-spamd


Best Regards.

Re: Help with RegEx Rule

[AK]

On 20/09/15 01:30, Benny Pedersen wrote:

On September 19, 2015 4:52:30 PM AK  wrote:


= Start Rule Block =
rawbody __MANY_PERIODS_1 ALL =~ /(^\.\n){5,}/


remove ALL =~, my own rawbody rules dont have it


Still no joy after removal.  However, at least the rule now hits if I 
replace:


/(^\.\n){5,}/

with

/(^\.\n)*/

But that looks like it might bring about some FPs.  Any other suggestions?


Regards,
ak.

Re: Help with RegEx Rule

[Dave Funk]

On Sun, 20 Sep 2015, AK wrote:


Hi all.

I'm getting hit with lots of JUNK mail that has multiple lines with just a 
'.' on several lines [0].  Most of the JUNK email has at least 5 and at most 
10 lines (so far) with just this '.' character somewhere in the middle of the 
message.


I've copied the message source to RegexBuddy [1] and have been able to come 
up with a regex that matches what I want using the Perl 5.20 engine:


(^\.\n){5,}

However, adding this rule to /etc/spamassassin/local.cf doesn't hit at all 
when I run it against my test message as follows:


= Start Rule Block =
rawbody __MANY_PERIODS_1 ALL =~ /(^\.\n){5,}/
meta MANY_PERIODS __MANY_PERIODS_1
score MANY_PERIODS 2.0
describe MANY_PERIODS JUNK mail with several lines that contain single dot
= End Rule Block =

= Begin Test Command =
spamassassin -L -t test.msg
= End Test Command =


Please help me understand what I'm doing wrong as this is my first attempt at 
creating a rule.  Previously I've just copied and pasted what I've found here 
in the forums, but this time I'm trying to do it myself but failing.



Regards,
ak.


SA does some interesting pre-processing on mail messages before applying 
rules, so you need to understand that.


Try this:

 rawbody T__LOCAL_MANY_PERIODS/\n(?:\.\n){5}?/
 describe T__LOCAL_MANY_PERIODS   Many lines with just a single "dot"

Notes:
1) Due to SA pre-processing collapsing body into one long line, cannot 
match on '^' repeatedly, need to look for '\n' as line break indicator.

Find start of a line and then following repeats of ".\n"
2) use '(?:' as grouping optimization unless you care about capture.
3) for terminal match clause use '{5}' not '{5,}' as we're done as soon
as we see at least 5 matches, don't care if there are more.
4) use "non-greedy" match quantifier '}?' look for first hit on that 
pattern and don't try to go for more.


Un-optimised pattern: /\n(\.\n){5}/

Note use of "testing" rule name format, that "T_". remove the leading 'T' 
to make it into a silent rule for combining with metas.


Personal convention; I interpolate '_LOCAL_' ( or '_L_') in locally 
created rule names to distinguish them for debugging. And then when things 
don't work as expected (EG: FPs) it helps to determine if the problem is 
self-inflicted.


Final note; now that we've discussed this spam sign, it will probably 
become useless as spammers follow this list and mutate their crap 
accordingly to dodge our rules. ;(


--
Dave Funk  University of Iowa
College of Engineering
319/335-5751   FAX: 319/384-0549   1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include 
Better is not better, 'standard' is better. B{

Re: Help with RegEx Rule

[Dave Funk]

On Sun, 20 Sep 2015, AK wrote:

[..snip..]
Still no joy after removal.  However, at least the rule now hits if I 
replace:


/(^\.\n){5,}/

with

/(^\.\n)*/

But that looks like it might bring about some FPs.  Any other suggestions?


Do you realize that rule will -always- fire on -any- message?
The '*' repeat operator is "zero or more" instances.
So that pattern degenerates to // which will match everything.

Guaranteed FP generator.

--
Dave Funk  University of Iowa
College of Engineering
319/335-5751   FAX: 319/384-0549   1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include 
Better is not better, 'standard' is better. B{