RE: SA frequently skipping rules
Thanks to Stuart and Daryl for your responses. I think I need to ask a basic question that I'm sure is a FAQ somewhere that I haven't located yet (honestly I've hunted!). How do I run a message through the spamassassin command line to get the score results on the screen? I tried saving the email and running spamassassin messagename -d spamassassin messagename -D -d and a few other variations but the results don't show any scored headers. BTW, thanks for the explanation on UNPARSEABLE_RELAY. I was thinking maybe the headers were scrambled so that SA tried to parse but gave up. That obviously isn't the case and not the reason I'm having difficulties. Once I can test select emails by running them back through to compare scores, that will help. Thanks, Jim Smith -Original Message- From: Stuart Johnston [mailto:[EMAIL PROTECTED] Sent: Thursday, February 09, 2006 7:12 PM To: Jim Smith; users@spamassassin.apache.org Subject: Re: SA frequently skipping rules This message does not hit any naughty words rules for me either (tested 3.1.0 and 3.0.3). SA doesn't generally have rules that hit a single word. To avoid FPs, it is better to check for phrases and obfuscations. However, the message does hit BAYES_99 and several networks tests on my system giving it a score of 31.5. Of course, network tests do tend to work better when you are investigating why a message got through than when the message first hits your mail server. Jim Smith wrote: I'm getting lots of spam that are skipping rules. One that came in recently with lots of porn only got tagged for SORBS, NUMERIC HELO, and UNPARSEABLE RELAY (I don't know what unparseable relay means but seems like many emails have that lately). The full headers message (uncensored) of that example is at www.blarneystone.com/spam/spam.txt if that helps. If you look at it you can tell that it should have kicked off lots of porn tags but none were there and it sailed through with a 3.2 score. This has only happened since I upgraded to SA 3.1.0. I've run SA --lint -D without errors. I thought it might be some configuration left over from my older SA when I upgraded so I did a clean install on a new machine and still have the same issue with skipping of rules. BTW, I know the rules aren't missing from the installation because they show up in other emails. A sporadic problem... my favorite sigh. Any suggestions? Thanks, Jim Smith
RE: SA frequently skipping rules
I typically use spamassassin -D testmessage. Kris -Original Message- From: Jim Smith [mailto:[EMAIL PROTECTED] Sent: Friday, February 10, 2006 9:16 AM To: users@spamassassin.apache.org Subject: RE: SA frequently skipping rules Thanks to Stuart and Daryl for your responses. I think I need to ask a basic question that I'm sure is a FAQ somewhere that I haven't located yet (honestly I've hunted!). How do I run a message through the spamassassin command line to get the score results on the screen? I tried saving the email and running spamassassin messagename -d spamassassin messagename -D -d and a few other variations but the results don't show any scored headers. BTW, thanks for the explanation on UNPARSEABLE_RELAY. I was thinking maybe the headers were scrambled so that SA tried to parse but gave up. That obviously isn't the case and not the reason I'm having difficulties. Once I can test select emails by running them back through to compare scores, that will help. Thanks, Jim Smith -Original Message- From: Stuart Johnston [mailto:[EMAIL PROTECTED] Sent: Thursday, February 09, 2006 7:12 PM To: Jim Smith; users@spamassassin.apache.org Subject: Re: SA frequently skipping rules This message does not hit any naughty words rules for me either (tested 3.1.0 and 3.0.3). SA doesn't generally have rules that hit a single word. To avoid FPs, it is better to check for phrases and obfuscations. However, the message does hit BAYES_99 and several networks tests on my system giving it a score of 31.5. Of course, network tests do tend to work better when you are investigating why a message got through than when the message first hits your mail server. Jim Smith wrote: I'm getting lots of spam that are skipping rules. One that came in recently with lots of porn only got tagged for SORBS, NUMERIC HELO, and UNPARSEABLE RELAY (I don't know what unparseable relay means but seems like many emails have that lately). The full headers message (uncensored) of that example is at www.blarneystone.com/spam/spam.txt if that helps. If you look at it you can tell that it should have kicked off lots of porn tags but none were there and it sailed through with a 3.2 score. This has only happened since I upgraded to SA 3.1.0. I've run SA --lint -D without errors. I thought it might be some configuration left over from my older SA when I upgraded so I did a clean install on a new machine and still have the same issue with skipping of rules. BTW, I know the rules aren't missing from the installation because they show up in other emails. A sporadic problem... my favorite sigh. Any suggestions? Thanks, Jim Smith
RE: SA frequently skipping rules
Oops, I sent that too quick. It should be spamassassin -r testmessage. -Original Message- From: Jim Smith [mailto:[EMAIL PROTECTED] Sent: Friday, February 10, 2006 9:16 AM To: users@spamassassin.apache.org Subject: RE: SA frequently skipping rules Thanks to Stuart and Daryl for your responses. I think I need to ask a basic question that I'm sure is a FAQ somewhere that I haven't located yet (honestly I've hunted!). How do I run a message through the spamassassin command line to get the score results on the screen? I tried saving the email and running spamassassin messagename -d spamassassin messagename -D -d and a few other variations but the results don't show any scored headers. BTW, thanks for the explanation on UNPARSEABLE_RELAY. I was thinking maybe the headers were scrambled so that SA tried to parse but gave up. That obviously isn't the case and not the reason I'm having difficulties. Once I can test select emails by running them back through to compare scores, that will help. Thanks, Jim Smith -Original Message- From: Stuart Johnston [mailto:[EMAIL PROTECTED] Sent: Thursday, February 09, 2006 7:12 PM To: Jim Smith; users@spamassassin.apache.org Subject: Re: SA frequently skipping rules This message does not hit any naughty words rules for me either (tested 3.1.0 and 3.0.3). SA doesn't generally have rules that hit a single word. To avoid FPs, it is better to check for phrases and obfuscations. However, the message does hit BAYES_99 and several networks tests on my system giving it a score of 31.5. Of course, network tests do tend to work better when you are investigating why a message got through than when the message first hits your mail server. Jim Smith wrote: I'm getting lots of spam that are skipping rules. One that came in recently with lots of porn only got tagged for SORBS, NUMERIC HELO, and UNPARSEABLE RELAY (I don't know what unparseable relay means but seems like many emails have that lately). The full headers message (uncensored) of that example is at www.blarneystone.com/spam/spam.txt if that helps. If you look at it you can tell that it should have kicked off lots of porn tags but none were there and it sailed through with a 3.2 score. This has only happened since I upgraded to SA 3.1.0. I've run SA --lint -D without errors. I thought it might be some configuration left over from my older SA when I upgraded so I did a clean install on a new machine and still have the same issue with skipping of rules. BTW, I know the rules aren't missing from the installation because they show up in other emails. A sporadic problem... my favorite sigh. Any suggestions? Thanks, Jim Smith
RE: SA frequently skipping rules
Forgive me for not understanding the porn filtering capability of SA. I ran a new email (www.blarneystone.com/spam/spam2.txt) through the SA filter (I didn't munge the headers this time). Do I understand it that if an email like that was sent from a URL not yet blacklisted, it would be scored very low regardless of the high level of porn in it (I kicked it up a few notches to make it more obvious). Or is my SA scores for tagging porn messages just not functioning correctly? Thanks, Jim Smith -Original Message- From: Daryl C. W. O'Shea [mailto:[EMAIL PROTECTED] Sent: Thursday, February 09, 2006 6:47 PM To: Jim Smith Cc: users@spamassassin.apache.org Subject: Re: SA frequently skipping rules Jim Smith wrote: I'm getting lots of spam that are skipping rules. One that came in recently with lots of porn only got tagged for SORBS, NUMERIC HELO, and UNPARSEABLE RELAY (I don't know what unparseable relay means but seems like many emails have that lately). UNPARSEABLE_RELAY means that, wait for it, one of the relays in the message headers (Received: headers) weren't parseable. The full headers message (uncensored) of that example is at www.blarneystone.com/spam/spam.txt if that helps. Full headers? There's nothing left of those headers. That sample is useless header wise. If you look at it you can tell that it should have kicked off lots of porn tags but none were there and it sailed through with a 3.2 score. This has only happened since I upgraded to SA 3.1.0. I don't see a single thing in the body that should have hit any rules. Except for some URIDNSBL rules [1] that you may or may not be running, but nothing content wise. I've run SA --lint -D without errors. I thought it might be some configuration left over from my older SA when I upgraded so I did a clean install on a new machine and still have the same issue with skipping of rules. BTW, I know the rules aren't missing from the installation because they show up in other emails. A sporadic problem... my favorite sigh. Any suggestions? Sparodic, as in, if you scan it again it hits different rules? Daryl [1] My hits on the sample... Content analysis details: (11.2 points, 5.0 required) pts rule name description -- -- 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines 2.6 NO_DNS_FOR_FROMDNS: Envelope sender has no MX or A DNS records 1.1 URIBL_SBL Contains an URL listed in the SBL blocklist [URIs: otrfgrt.com] 3.4 URIBL_JP_SURBL Contains an URL listed in the JP SURBL blocklist [URIs: otrfgrt.com] 1.5 URIBL_WS_SURBL Contains an URL listed in the WS SURBL blocklist [URIs: otrfgrt.com] 2.6 URIBL_OB_SURBL Contains an URL listed in the OB SURBL blocklist [URIs: otrfgrt.com]
RE: SA frequently skipping rules
Title: RE: SA frequently skipping rules The problem with writing rules for this, is generating FPs. I mean, I get email slike that from my wife all the time. ;) I'm sure I'm not the only one who gets them. Well I hope other people don't get them from my wife. Anway, the presence of bad words like that doesn't really mean it is spam. So its kind of tough to nail it down. Heck, my ice hockey team email shave a lot more profanity then that :) Although none of my team members have asked my to tit screws them. Thanks to URIBL and SURBL we don't really worry about these much. Chris Santerre SysAdmin and SARE/URIBL ninja http://www.uribl.com http://www.rulesemporium.com -Original Message- From: Jim Smith [mailto:[EMAIL PROTECTED]] Sent: Friday, February 10, 2006 12:01 PM To: users@spamassassin.apache.org Subject: RE: SA frequently skipping rules Forgive me for not understanding the porn filtering capability of SA. I ran a new email (www.blarneystone.com/spam/spam2.txt) through the SA filter (I didn't munge the headers this time). Do I understand it that if an email like that was sent from a URL not yet blacklisted, it would be scored very low regardless of the high level of porn in it (I kicked it up a few notches to make it more obvious). Or is my SA scores for tagging porn messages just not functioning correctly? Thanks, Jim Smith -Original Message- From: Daryl C. W. O'Shea [mailto:[EMAIL PROTECTED]] Sent: Thursday, February 09, 2006 6:47 PM To: Jim Smith Cc: users@spamassassin.apache.org Subject: Re: SA frequently skipping rules Jim Smith wrote: I'm getting lots of spam that are skipping rules. One that came in recently with lots of porn only got tagged for SORBS, NUMERIC HELO, and UNPARSEABLE RELAY (I don't know what unparseable relay means but seems like many emails have that lately). UNPARSEABLE_RELAY means that, wait for it, one of the relays in the message headers (Received: headers) weren't parseable. The full headers message (uncensored) of that example is at www.blarneystone.com/spam/spam.txt if that helps. Full headers? There's nothing left of those headers. That sample is useless header wise. If you look at it you can tell that it should have kicked off lots of porn tags but none were there and it sailed through with a 3.2 score. This has only happened since I upgraded to SA 3.1.0. I don't see a single thing in the body that should have hit any rules. Except for some URIDNSBL rules [1] that you may or may not be running, but nothing content wise. I've run SA --lint -D without errors. I thought it might be some configuration left over from my older SA when I upgraded so I did a clean install on a new machine and still have the same issue with skipping of rules. BTW, I know the rules aren't missing from the installation because they show up in other emails. A sporadic problem... my favorite sigh. Any suggestions? Sparodic, as in, if you scan it again it hits different rules? Daryl [1] My hits on the sample... Content analysis details: (11.2 points, 5.0 required) pts rule name description -- -- 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines 2.6 NO_DNS_FOR_FROM DNS: Envelope sender has no MX or A DNS records 1.1 URIBL_SBL Contains an URL listed in the SBL blocklist [URIs: otrfgrt.com] 3.4 URIBL_JP_SURBL Contains an URL listed in the JP SURBL blocklist [URIs: otrfgrt.com] 1.5 URIBL_WS_SURBL Contains an URL listed in the WS SURBL blocklist [URIs: otrfgrt.com] 2.6 URIBL_OB_SURBL Contains an URL listed in the OB SURBL blocklist [URIs: otrfgrt.com]
Re: SA frequently skipping rules
Forgive me for not understanding the porn filtering capability of SA. I ran a new email (www.blarneystone.com/spam/spam2.txt) through the SA filter (I didn't munge the headers this time). Do I understand it that if an email like that was sent from a URL not yet blacklisted, it would be scored very low regardless of the high level of porn in it (I kicked it up a few notches to make it more obvious). Or is my SA scores for tagging porn messages just not functioning correctly? SA out of the box has realtively weak filter for porn words, and with good reason. SA is a **SPAM** filter, not a PORN filter. Just because porn comes in spams doesn't mean that it doesn't also come in non-spam. SA isn't a content filter out of the box. Of course, you can turn SA into a porn filter, and it will filter out all porn for you, whether it is spam or not. If you are a public library or government-funded school or other instutition based on the concept of freedom of informaiton to the public, then in general your policy will be to do this. Pontification aside, go get yourself the sare_adult.cf ruleset. It will add points for a lot of this stuff. It will NOT add points for all of it. For instance, breast is not considered a porn word any more, since it turned out some women didn't consider discussion of breast cancer recovery to be pornographic. Also be careful of writing the usual ill thought out and draconian rules youself like body MY_PORN_1/cock/i scoreMY_PORN_1100# eliminate evil words! That works just fine. Unless you maybe have a user or a client named John Babcock. Loren
Re: SA frequently skipping rules
Jim Smith wrote: I'm getting lots of spam that are skipping rules. One that came in recently with lots of porn only got tagged for SORBS, NUMERIC HELO, and UNPARSEABLE RELAY (I don't know what unparseable relay means but seems like many emails have that lately). UNPARSEABLE_RELAY means that, wait for it, one of the relays in the message headers (Received: headers) weren't parseable. The full headers message (uncensored) of that example is at www.blarneystone.com/spam/spam.txt if that helps. Full headers? There's nothing left of those headers. That sample is useless header wise. If you look at it you can tell that it should have kicked off lots of porn tags but none were there and it sailed through with a 3.2 score. This has only happened since I upgraded to SA 3.1.0. I don't see a single thing in the body that should have hit any rules. Except for some URIDNSBL rules [1] that you may or may not be running, but nothing content wise. I've run SA --lint -D without errors. I thought it might be some configuration left over from my older SA when I upgraded so I did a clean install on a new machine and still have the same issue with skipping of rules. BTW, I know the rules aren't missing from the installation because they show up in other emails. A sporadic problem... my favorite sigh. Any suggestions? Sparodic, as in, if you scan it again it hits different rules? Daryl [1] My hits on the sample... Content analysis details: (11.2 points, 5.0 required) pts rule name description -- -- 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines 2.6 NO_DNS_FOR_FROMDNS: Envelope sender has no MX or A DNS records 1.1 URIBL_SBL Contains an URL listed in the SBL blocklist [URIs: otrfgrt.com] 3.4 URIBL_JP_SURBL Contains an URL listed in the JP SURBL blocklist [URIs: otrfgrt.com] 1.5 URIBL_WS_SURBL Contains an URL listed in the WS SURBL blocklist [URIs: otrfgrt.com] 2.6 URIBL_OB_SURBL Contains an URL listed in the OB SURBL blocklist [URIs: otrfgrt.com]
Re: SA frequently skipping rules
This message does not hit any naughty words rules for me either (tested 3.1.0 and 3.0.3). SA doesn't generally have rules that hit a single word. To avoid FPs, it is better to check for phrases and obfuscations. However, the message does hit BAYES_99 and several networks tests on my system giving it a score of 31.5. Of course, network tests do tend to work better when you are investigating why a message got through than when the message first hits your mail server. Jim Smith wrote: I'm getting lots of spam that are skipping rules. One that came in recently with lots of porn only got tagged for SORBS, NUMERIC HELO, and UNPARSEABLE RELAY (I don't know what unparseable relay means but seems like many emails have that lately). The full headers message (uncensored) of that example is at www.blarneystone.com/spam/spam.txt if that helps. If you look at it you can tell that it should have kicked off lots of porn tags but none were there and it sailed through with a 3.2 score. This has only happened since I upgraded to SA 3.1.0. I've run SA --lint -D without errors. I thought it might be some configuration left over from my older SA when I upgraded so I did a clean install on a new machine and still have the same issue with skipping of rules. BTW, I know the rules aren't missing from the installation because they show up in other emails. A sporadic problem... my favorite sigh. Any suggestions? Thanks, Jim Smith
RE: SA frequently skipping rules
Jim Smith wrote: I'm getting lots of spam that are skipping rules. One that came in recently with lots of porn only got tagged for SORBS, NUMERIC HELO, and UNPARSEABLE RELAY (I don't know what unparseable relay means but seems like many emails have that lately). The full headers message (uncensored) of that example is at www.blarneystone.com/spam/spam.txt if that helps. I think that this is the unparseable relay: Received: from mail.x.edu by xxx.xx.xxx.xxx (8.12.11/8.12.11) with ESMTP id 2XaVd6sLk8ikAV for [EMAIL PROTECTED]; Wed, 8 Feb 2006 08:44:46 -0800 Notice there's no indication of what IP address the mail was received from. (by != from.) I would bet heavily that this header was spoofed. The only headers you can trust are the ones added by servers you know... in this case, it looks like the top two Received: headers are by trustworthy servers. -- Matthew.van.Eerde (at) hbinc.com 805.964.4554 x902 Hispanic Business Inc./HireDiversity.com Software Engineer
