Re: Lots of scam messages getting through SA
Fixed this by upgrading SA to 3.2.4
Lots of scam messages getting through SA
I have started, over the last few months, getting a lot of plain text scam messages (Nigerian type scams, lottery wins etc etc). Previously I had almost none of these. Unfortunately I'd need to send rather a lot of information about my configs, and log files to help, but can anybody point me in the right direction. I run sa-update every night. I do quite extensive filtering before sending messages to spamassassin - I use xbl.spamhaus.org to block messages and I also use selective greylisting and clam antivirus (which detects some scams). Maybe I'm not getting enough spam to train SA adequately (I train based on my spam and ham folder weekly). Sorry about the lack of detail, but I'm hoping somebody might have a simple answer.
Re: Lots of scam messages getting through SA
Robert S wrote: I have started, over the last few months, getting a lot of plain text scam messages (Nigerian type scams, lottery wins etc etc). Previously I had almost none of these. Unfortunately I'd need to send rather a lot of information about my configs, and log files to help, but can anybody point me in the right direction. I run sa-update every night. The first step is to make a copy of such scams (full headers and body) available (on pastebin for example). I do quite extensive filtering before sending messages to spamassassin - I use xbl.spamhaus.org to block messages and I also use selective greylisting and clam antivirus (which detects some scams). Maybe I'm not getting enough spam to train SA adequately (I train based on my spam and ham folder weekly). Sorry about the lack of detail, but I'm hoping somebody might have a simple answer.
Re: Lots of scam messages getting through SA
On Fri, 2008-03-07 at 22:41 +1100, Robert S wrote: I have started, over the last few months, getting a lot of plain text scam messages (Nigerian type scams, lottery wins etc etc). Previously I had almost none of these. Unfortunately I'd need to send rather a lot of information about my configs, and log files to help, but can anybody point me in the right direction. I run sa-update every night. I do quite extensive filtering before sending messages to spamassassin - I use xbl.spamhaus.org to block messages and I also use selective greylisting and clam antivirus (which detects some scams). Maybe I'm not getting enough spam to train SA adequately (I train based on my spam and ham folder weekly). Sorry about the lack of detail, but I'm hoping somebody might have a simple answer. Yes .. Some of these keep trickling tru :-( But you should be able to catch most of them Have you added the sought rules from http://taint.org/2007/08/15/004348a.html With these rules and my custom rules I catch 99% of these But I keep getting some 2-5 daily complaints yet from customers The problem I have noticed is that these spams usually come from hijacked accounts. Hence they come from legitimate mail servers and usually get thru clean from ip reputation filters. So your *.spamhaus *.spamcop dont work here A frighteningly large number of email users , use very simple passwords ( like password, welcome1 , hello , pass .. ) So spammers just guess these passwords. Use authenticated sessions and bombard spams. By the time the admin realizes this 1000's of spams have already been sent But ultimately this boils down to end user education. Recipients must realize that no one from Africa is going to transfer all the millions of dollars in an unknown account , or there is nothing called as a national lottery in the united Kingdom Thanks Ram
Re: Lots of scam messages getting through SA
In article [EMAIL PROTECTED], ram [EMAIL PROTECTED] writes But ultimately this boils down to end user education. Recipients must realize that no one from Africa is going to transfer all the millions of dollars in an unknown account , or there is nothing called as a national lottery in the united Kingdom Why do people keep saying this? http://www.national-lottery.co.uk/ Yes, it's unlikely someone who lives across the globe and never purchased a ticket would win, but it's a legit thing. In fact Camelot (the company behind it) run the EuroMillions lottery in 9 different countries apparently. Education is useful but if someone can see you're wrong in 30 seconds they'll be less likely to believe you're right about dethroned kings sending money. Kevin
Re: Lots of scam messages getting through SA
Have you added the sought rules from http://taint.org/2007/08/15/004348a.html With these rules and my custom rules I catch 99% of these But I keep getting some 2-5 daily complaints yet from customers I think you're right about the source of these scams. I've installed this according to the instructions at taint.org. Presumably I don't need to do anything else to make this work. One problem with this is that the rules won't compile any more. Any suggestions? # sa-compile [3635] info: generic: base extraction starting. this can take a while... [3635] info: generic: extracting from rules of type body_0 100% [===] 82.00 rules/sec 00m16s DONE 100% [===] 55.06 bases/sec 00m41s DONE [3635] info: body_0: 1865 base strings extracted in 58 seconds [3635] info: generic: extracting from rules of type body_500 100% [===] 70.76 rules/sec 00m00s DONE 100% [===] 264.09 bases/sec 00m00s DONE [3635] info: body_500: 2 base strings extracted in 0 seconds cd /tmp/.spamassassin3635vg5K00tmp cd Mail-SpamAssassin-CompiledRegexps-body_0 Wide character in print at /usr/bin/sa-compile line 379, $fh line 1168. re2c -i -b -o scanner1.c scanner1.re re2c -i -b -o scanner2.c scanner2.re re2c: error: line 42, column 27: can't find symbol command failed! at /usr/bin/sa-compile line 282, $fh line 2964.
Re: These are getting through SA...
Phil, Bill, Mark, I patched Dns.pm but this didn't resolve the issue for me. You can test with the sample messages I posted to bugzilla: http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5506 I was getting this sort of symptom without using Botnet. It's almost as if something's deadlocking somewhere in SA (until the timeout kicks in). If the time spent by Razor+dcc+Botnet+(not sure what else) is longer than rbl_timeout, then replies to RBL queries are thrown away by mistake. There is now an additional patch at: http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511 which should fix this. Mark
Re: These are getting through SA...
Mark Martinec wrote the following on 6/15/2007 3:36 AM -0800: Phil, Bill, Mark, I patched Dns.pm but this didn't resolve the issue for me. You can test with the sample messages I posted to bugzilla: http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5506 I was getting this sort of symptom without using Botnet. It's almost as if something's deadlocking somewhere in SA (until the timeout kicks in). If the time spent by Razor+dcc+Botnet+(not sure what else) is longer than rbl_timeout, then replies to RBL queries are thrown away by mistake. There is now an additional patch at: http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511 which should fix this. Mark Mark, thanks for the patches. However, even with both Dns.pm patches applied, unless I set rbl_timeout to a high enough time interval, SA still misses the URIBL test results in the sample messages I posted in bugzilla 5506. For example, if I set rbl_timeout 15, I get no URIBL hit results. However, if I set rbl_timeout 60, I do get URIBL hit results. This is because the 60 interval is long enough for Botnet's (in this case) RDNS test to timeout before the rbl_timeout interval. Bill
Re: These are getting through SA...
Bill, There is now an additional patch at: http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511 which should fix this. Mark, thanks for the patches. However, even with both Dns.pm patches applied, unless I set rbl_timeout to a high enough time interval, SA still misses the URIBL test results in the sample messages I posted in bugzilla 5506. For example, if I set rbl_timeout 15, I get no URIBL hit results. However, if I set rbl_timeout 60, I do get URIBL hit results. This is because the 60 interval is long enough for Botnet's (in this case) RDNS test to timeout before the rbl_timeout interval. Don't know. I was using your test case (test1.txt) to cause a Botnet hang, had rbl_timeout at 15, and I did get the RBL hits. Try spamassassin from a command line with -t -D. If you applied both patches, there should be something like the following in the debug output. 19:26:55.558 8.010 0.000 [13051] dbg: Botnet: no trusted relays 19:26:55.558 8.010 0.000 [13051] dbg: Botnet: get_relay didn't find RDNS ...a bug pause here... 19:27:24.558 37.010 28.999 [13051] dbg: Botnet: IP is '66.17.235.109' 19:27:24.558 37.010 0.001 [13051] dbg: Botnet: RDNS is '' 19:27:24.559 37.011 0.001 [13051] dbg: Botnet: HELO is 'xx' 19:27:24.559 37.011 0.000 [13051] dbg: Botnet: BADDNS miss ... 19:27:26.269 38.721 0.002 [13051] dbg: rules: running meta tests; score so far=12.753 19:27:26.269 38.721 0.001 [13051] dbg: rules: compiled meta tests 19:27:26.271 38.723 0.002 [13051] dbg: check: running tests for priority: 500 19:27:26.274 38.726 0.002 [13051] dbg: async: select found 1 socks ready 19:27:26.275 38.727 0.001 [13051] dbg: uridnsbl: query for xxx took 35 seconds to look up (dob.sibl.support-intelligence.net:xxx) ... 19:27:26.281 38.733 0.001 [13051] dbg: async: queries completed: 15 started: 2 19:27:26.283 38.735 0.001 [13051] dbg: async: queries active: at Fri Jun 15 19:27:26 2007 vvv 19:27:26.283 38.735 0.001 [13051] dbg: dns: harvest_dnsbl_queries: on extended time, overdue by 30.000 s, still 1.200 s 19:27:26.295 38.747 0.011 [13051] dbg: async: select found 1 socks ready 19:27:26.297 38.749 0.002 [13051] dbg: async: queries completed: 1 started: 2 19:27:26.297 38.749 0.000 [13051] dbg: async: queries active: URI-A=1 at Fri ... 0.1 RDNS_NONE Delivered to trusted network by a host with no rDNS 3.4 HEADER_SPAMBulk email fingerprint (header-based) found 0.1 BOTNET Relay might be a spambot or virusbot [botnet0.7,ip=66.17.235.109,nordns] 0.0 DKIM_POLICY_SIGNSOME Domain Keys Identified Mail: policy says domain signs some mails 0.0 BOTNET_NORDNS Relay's IP address has no PTR record [botnet_nordns,ip=66.17.235.109] 1.2 HTML_IMAGE_ONLY_28 BODY: HTML: images with 2400-2800 bytes of words 0.2 HTML_MESSAGE BODY: HTML included in message 3.0 BAYES_95 BODY: Bayesian spam probability is 95 to 99% [score: 0.9658] 1.5 MIME_HTML_ONLY BODY: Message only has text/html MIME parts 0.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/) 2.8 DCC_CHECK Listed in DCC (http://rhyolite.com/anti-spam/dcc/) 2.0 URIBL_BLACKContains an URL listed in the URIBL blacklist [URIs: xxx] 1.5 URIBL_OB_SURBL Contains an URL listed in the OB SURBL blocklist [URIs: xxx] 0.0 DIGEST_MULTIPLEMessage hits more than one network digest check 0.5 BOTNET_OTHER BOTNET_OTHER -1.2 AWLAWL: From: address is in the auto white-list (I had to strike the uri in the log to be able to post the message :) Mark
Re: These are getting through SA...
...a bug pause here... bug - big (29 seconds)
Re: These are getting through SA...
Mark Martinec wrote the following on 6/15/2007 10:41 AM -0800: Bill, There is now an additional patch at: http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511 which should fix this. Mark, thanks for the patches. However, even with both Dns.pm patches applied, unless I set rbl_timeout to a high enough time interval, SA still misses the URIBL test results in the sample messages I posted in bugzilla 5506. For example, if I set rbl_timeout 15, I get no URIBL hit results. However, if I set rbl_timeout 60, I do get URIBL hit results. This is because the 60 interval is long enough for Botnet's (in this case) RDNS test to timeout before the rbl_timeout interval. Don't know. I was using your test case (test1.txt) to cause a Botnet hang, had rbl_timeout at 15, and I did get the RBL hits. Try spamassassin from a command line with -t -D. If you applied both patches, there should be something like the following in the debug output. 19:26:55.558 8.010 0.000 [13051] dbg: Botnet: no trusted relays 19:26:55.558 8.010 0.000 [13051] dbg: Botnet: get_relay didn't find RDNS ...a bug pause here... 19:27:24.558 37.010 28.999 [13051] dbg: Botnet: IP is '66.17.235.109' 19:27:24.558 37.010 0.001 [13051] dbg: Botnet: RDNS is '' 19:27:24.559 37.011 0.001 [13051] dbg: Botnet: HELO is 'xx' 19:27:24.559 37.011 0.000 [13051] dbg: Botnet: BADDNS miss ... 19:27:26.269 38.721 0.002 [13051] dbg: rules: running meta tests; score so far=12.753 19:27:26.269 38.721 0.001 [13051] dbg: rules: compiled meta tests 19:27:26.271 38.723 0.002 [13051] dbg: check: running tests for priority: 500 19:27:26.274 38.726 0.002 [13051] dbg: async: select found 1 socks ready 19:27:26.275 38.727 0.001 [13051] dbg: uridnsbl: query for xxx took 35 seconds to look up (dob.sibl.support-intelligence.net:xxx) ... 19:27:26.281 38.733 0.001 [13051] dbg: async: queries completed: 15 started: 2 19:27:26.283 38.735 0.001 [13051] dbg: async: queries active: at Fri Jun 15 19:27:26 2007 vvv 19:27:26.283 38.735 0.001 [13051] dbg: dns: harvest_dnsbl_queries: on extended time, overdue by 30.000 s, still 1.200 s 19:27:26.295 38.747 0.011 [13051] dbg: async: select found 1 socks ready 19:27:26.297 38.749 0.002 [13051] dbg: async: queries completed: 1 started: 2 19:27:26.297 38.749 0.000 [13051] dbg: async: queries active: URI-A=1 at Fri ... 0.1 RDNS_NONE Delivered to trusted network by a host with no rDNS 3.4 HEADER_SPAMBulk email fingerprint (header-based) found 0.1 BOTNET Relay might be a spambot or virusbot [botnet0.7,ip=66.17.235.109,nordns] 0.0 DKIM_POLICY_SIGNSOME Domain Keys Identified Mail: policy says domain signs some mails 0.0 BOTNET_NORDNS Relay's IP address has no PTR record [botnet_nordns,ip=66.17.235.109] 1.2 HTML_IMAGE_ONLY_28 BODY: HTML: images with 2400-2800 bytes of words 0.2 HTML_MESSAGE BODY: HTML included in message 3.0 BAYES_95 BODY: Bayesian spam probability is 95 to 99% [score: 0.9658] 1.5 MIME_HTML_ONLY BODY: Message only has text/html MIME parts 0.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/) 2.8 DCC_CHECK Listed in DCC (http://rhyolite.com/anti-spam/dcc/) 2.0 URIBL_BLACKContains an URL listed in the URIBL blacklist [URIs: xxx] 1.5 URIBL_OB_SURBL Contains an URL listed in the OB SURBL blocklist [URIs: xxx] 0.0 DIGEST_MULTIPLEMessage hits more than one network digest check 0.5 BOTNET_OTHER BOTNET_OTHER -1.2 AWLAWL: From: address is in the auto white-list (I had to strike the uri in the log to be able to post the message :) Mark Mark, I'm really sorry if I caused you to unnecessarily spin your wheels on this, I applied the second patch to the wrong SA version of Dns.pm. The patches work fine. Again, my sincere apologies... :-( Bill
RE: These are getting through SA...
Bill, The problem is that Botnet uses Net::DNS::Resolver's default retry and timeout values, which are way too high. Spamassassin's DnsResolver.pm uses these values: udp_timeout:3 tcp_timeout:3 retrans:0 retry:1 try export RES_OPTIONS=udp_timeout:3 tcp_timeout:3 retrans:0 retry:1 before invoking spamassassin and see what happens. The proper solution would be to fix Botnet.pm to use sensible values for these parameters instead of the defaults. Cheers, Phil -Original Message- From: Bill Landry [mailto:[EMAIL PROTECTED] Sent: 15 June 2007 18:18 To: users@spamassassin.apache.org Subject: Re: These are getting through SA... Mark, thanks for the patches. However, even with both Dns.pm patches applied, unless I set rbl_timeout to a high enough time interval, SA still misses the URIBL test results in the sample messages I posted in bugzilla 5506. For example, if I set rbl_timeout 15, I get no URIBL hit results. However, if I set rbl_timeout 60, I do get URIBL hit results. This is because the 60 interval is long enough for Botnet's (in this case) RDNS test to timeout before the rbl_timeout interval. Bill
Re: These are getting through SA...
Randal, Phil wrote the following on 6/15/2007 2:08 PM -0800: Bill, The problem is that Botnet uses Net::DNS::Resolver's default retry and timeout values, which are way too high. Spamassassin's DnsResolver.pm uses these values: udp_timeout:3 tcp_timeout:3 retrans:0 retry:1 try export RES_OPTIONS=udp_timeout:3 tcp_timeout:3 retrans:0 retry:1 before invoking spamassassin and see what happens. The proper solution would be to fix Botnet.pm to use sensible values for these parameters instead of the defaults. Cheers, Phil Hmmm, once I patched the correct SA version Dns.pm file, Mark's patches worked fine. However, perhaps my error caused Mark to find a bug, as noted by his follow-up e-mail, which might have gone undetected otherwise. :-) Also, I'm not sure if John Rudd is still supporting Botnet or not, since I have sent him 3 e-mails to the address listed in Botnet.pm off-list over the past week about this, and asked him if he would consider adding user configurable timeout values, but have not received any responses back from him. So far so good with Mark's patches - although I am awaiting his follow-up regarding a possible bug... Bill
Re: These are getting through SA...
Bill, Hmmm, once I patched the correct SA version Dns.pm file, Mark's patches worked fine. However, perhaps my error caused Mark to find a bug, as noted by his follow-up e-mail, which might have gone undetected otherwise. :-) Indeed, thanks! (but there were two other similar reports as well) Also, I'm not sure if John Rudd is still supporting Botnet or not, since I have sent him 3 e-mails to the address listed in Botnet.pm off-list over the past week about this, and asked him if he would consider adding user configurable timeout values, but have not received any responses back from him. It would definitely be worth shortening a dns timeout limit in Botnet, or make it configurable (or both). So far so good with Mark's patches - although I am awaiting his follow-up regarding a possible bug... Not sure I understand this. My fixes make SA more robust when plugins misbehave. The Botnet problem still causes the mail processing to take half a minute, but now without ill-effects on other checks. Mark
Re: These are getting through SA...
Mark Martinec wrote the following on 6/15/2007 2:34 PM -0800: So far so good with Mark's patches - although I am awaiting his follow-up regarding a possible bug... Not sure I understand this. My fixes make SA more robust when plugins misbehave. The Botnet problem still causes the mail processing to take half a minute, but now without ill-effects on other checks. Mark I was referring to your e-mail to the list just previous to this one where you stated: = ...a bug pause here... bug - big (29 seconds) = I misunderstood this to mean that you had found a bug in one of your patches. Please disregard. And thanks again for the patches! They do indeed make SA more robust by resolving the issue of lost DNS query results due to high timeout thresholds in plugins. Bill
RE: These are getting through SA...
On Fri, 2007-06-15 at 22:08 +0100, Randal, Phil wrote:
Bill,
The problem is that Botnet uses Net::DNS::Resolver's default retry and
timeout values, which are way too high.
Spamassassin's DnsResolver.pm uses these values:
udp_timeout:3
tcp_timeout:3
retrans:0
retry:1
And a few others... Might as well be completely consistent. Try this
patch:
--- Botnet.pm.orig 2007-06-15 16:47:33.0 -0500
+++ Botnet.pm 2007-06-15 16:52:13.0 -0500
@@ -703,7 +703,16 @@
($type =~ /^(?:A|MX)$/)
(defined $max)
($max =~ /^-?\d+$/) ) {
- $resolver = Net::DNS::Resolver-new();
+ $resolver = Net::DNS::Resolver-new(
+ udp_timeout = 3,
+ tcp_timeout = 3,
+ retrans = 0,
+ retry = 1,
+ persistent_tcp=0,
+ persistent_udp=0,
+ dnsrch=0,
+ defnames=0,
+ );
if ($query = $resolver-search($name, $type)) {
# found matches
$i = 0;
@@ -826,7 +835,18 @@
sub get_rdns {
my ($ip) = @_;
my ($query, @answer, $rr);
- my $resolver = Net::DNS::Resolver-new();
+ my $resolver = Net::DNS::Resolver-new(
+ udp_timeout = 3,
+ tcp_timeout = 3,
+ retrans = 0,
+ retry = 1,
+ persistent_tcp=0,
+ persistent_udp=0,
+ dnsrch=0,
+ defnames=0,
+ );
+ if ($query = $resolver-search($name, $type)) {
+ # found matches
my $name = ;
if ($query = $resolver-query($ip, 'PTR', 'IN')) {
--
Daniel J McDonald, CCIE # 2495, CISSP # 78281, CNX
Austin Energy
http://www.austinenergy.com
Re: These are getting through SA...
Bill Landry wrote: Also, I'm not sure if John Rudd is still supporting Botnet or not, since I have sent him 3 e-mails to the address listed in Botnet.pm off-list over the past week about this, and asked him if he would consider adding user configurable timeout values, but have not received any responses back from him. I am, but I haven't received those email messages from you.
Re: These are getting through SA...
John Rudd wrote: Bill Landry wrote: Also, I'm not sure if John Rudd is still supporting Botnet or not, since I have sent him 3 e-mails to the address listed in Botnet.pm off-list over the past week about this, and asked him if he would consider adding user configurable timeout values, but have not received any responses back from him. I am, but I haven't received those email messages from you. Looks like they were marked as spam, because the site you mentioned was listed on 3 different URIBL's. I'll look into the options you presented.
Re: These are getting through SA...
John Rudd wrote the following on 6/15/2007 3:00 PM -0800: Bill Landry wrote: Also, I'm not sure if John Rudd is still supporting Botnet or not, since I have sent him 3 e-mails to the address listed in Botnet.pm off-list over the past week about this, and asked him if he would consider adding user configurable timeout values, but have not received any responses back from him. I am, but I haven't received those email messages from you. John, could they have gotten caught in you spam filters possibly? I sent messages to you at the same e-mail address you used for posting to this list, two on 6/8/07 and one on 6/12/07. They would have come from the same e-mail address I am using here to post this message to the sa-users list. I could try resending them to you if you would like...? Bill
Re: These are getting through SA...
Daniel J McDonald wrote the following on 6/15/2007 2:54 PM -0800:
On Fri, 2007-06-15 at 22:08 +0100, Randal, Phil wrote:
Bill,
The problem is that Botnet uses Net::DNS::Resolver's default retry and
timeout values, which are way too high.
Spamassassin's DnsResolver.pm uses these values:
udp_timeout:3
tcp_timeout:3
retrans:0
retry:1
And a few others... Might as well be completely consistent. Try this
patch:
--- Botnet.pm.orig 2007-06-15 16:47:33.0 -0500
+++ Botnet.pm 2007-06-15 16:52:13.0 -0500
@@ -703,7 +703,16 @@
($type =~ /^(?:A|MX)$/)
(defined $max)
($max =~ /^-?\d+$/) ) {
- $resolver = Net::DNS::Resolver-new();
+ $resolver = Net::DNS::Resolver-new(
+ udp_timeout = 3,
+ tcp_timeout = 3,
+ retrans = 0,
+ retry = 1,
+ persistent_tcp=0,
+ persistent_udp=0,
+ dnsrch=0,
+ defnames=0,
+ );
if ($query = $resolver-search($name, $type)) {
# found matches
$i = 0;
@@ -826,7 +835,18 @@
sub get_rdns {
my ($ip) = @_;
my ($query, @answer, $rr);
- my $resolver = Net::DNS::Resolver-new();
+ my $resolver = Net::DNS::Resolver-new(
+ udp_timeout = 3,
+ tcp_timeout = 3,
+ retrans = 0,
+ retry = 1,
+ persistent_tcp=0,
+ persistent_udp=0,
+ dnsrch=0,
+ defnames=0,
+ );
+ if ($query = $resolver-search($name, $type)) {
+ # found matches
my $name = ;
if ($query = $resolver-query($ip, 'PTR', 'IN')) {
Daniel, here is a snippet of my debug output with this patch applied to
Botnet.pm, version 0.7:
[23898] warn: plugin: failed to parse plugin
/etc/mail/spamassassin/Botnet.pm: Global symbol $name requires
explicit package name at /etc/mail/spamassassin/Botnet.pm line 848.
[23898] warn: Global symbol $type requires explicit package name at
/etc/mail/spamassassin/Botnet.pm line 848.
[23898] warn: Missing right curly or square bracket at
/etc/mail/spamassassin/Botnet.pm line 875, at end of line
[23898] warn: syntax error at /etc/mail/spamassassin/Botnet.pm line 875,
at EOF
[23898] warn: Compilation failed in require at
/usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PluginHandler.pm line 97.
Bill
Re: These are getting through SA...
On Fri, 2007-06-15 at 15:27 -0700, Bill Landry wrote:
Daniel J McDonald wrote the following on 6/15/2007 2:54 PM -0800:
On Fri, 2007-06-15 at 22:08 +0100, Randal, Phil wrote:
And a few others... Might as well be completely consistent. Try this
patch:
--- Botnet.pm.orig 2007-06-15 16:47:33.0 -0500
+++ Botnet.pm 2007-06-15 16:52:13.0 -0500
Daniel, here is a snippet of my debug output with this patch applied to
Botnet.pm, version 0.7:
[23898] warn: plugin: failed to parse plugin
/etc/mail/spamassassin/Botnet.pm: Global symbol $name requires
explicit package name at /etc/mail/spamassassin/Botnet.pm line 848.
[23898] warn: Global symbol $type requires explicit package name at
/etc/mail/spamassassin/Botnet.pm line 848.
[23898] warn: Missing right curly or square bracket at
/etc/mail/spamassassin/Botnet.pm line 875, at end of line
[23898] warn: syntax error at /etc/mail/spamassassin/Botnet.pm line 875,
at EOF
[23898] warn: Compilation failed in require at
/usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PluginHandler.pm line 97.
Oops, yanked an entra line. Try this one instead:
--- Botnet.pm.orig 2007-06-15 16:47:33.0 -0500
+++ Botnet.pm 2007-06-15 17:35:33.0 -0500
@@ -703,7 +703,16 @@
($type =~ /^(?:A|MX)$/)
(defined $max)
($max =~ /^-?\d+$/) ) {
- $resolver = Net::DNS::Resolver-new();
+ $resolver = Net::DNS::Resolver-new(
+ udp_timeout = 3,
+ tcp_timeout = 3,
+ retrans = 0,
+ retry = 1,
+ persistent_tcp=0,
+ persistent_udp=0,
+ dnsrch=0,
+ defnames=0,
+ );
if ($query = $resolver-search($name, $type)) {
# found matches
$i = 0;
@@ -826,7 +835,16 @@
sub get_rdns {
my ($ip) = @_;
my ($query, @answer, $rr);
- my $resolver = Net::DNS::Resolver-new();
+ my $resolver = Net::DNS::Resolver-new(
+ udp_timeout = 3,
+ tcp_timeout = 3,
+ retrans = 0,
+ retry = 1,
+ persistent_tcp=0,
+ persistent_udp=0,
+ dnsrch=0,
+ defnames=0,
+ );
my $name = ;
if ($query = $resolver-query($ip, 'PTR', 'IN')) {
Bill
Re: These are getting through SA...
Daniel J McDonald wrote the following on 6/15/2007 3:37 PM -0800:
On Fri, 2007-06-15 at 15:27 -0700, Bill Landry wrote:
Daniel J McDonald wrote the following on 6/15/2007 2:54 PM -0800:
On Fri, 2007-06-15 at 22:08 +0100, Randal, Phil wrote:
And a few others... Might as well be completely consistent. Try this
patch:
--- Botnet.pm.orig 2007-06-15 16:47:33.0 -0500
+++ Botnet.pm 2007-06-15 16:52:13.0 -0500
Daniel, here is a snippet of my debug output with this patch applied to
Botnet.pm, version 0.7:
[23898] warn: plugin: failed to parse plugin
/etc/mail/spamassassin/Botnet.pm: Global symbol $name requires
explicit package name at /etc/mail/spamassassin/Botnet.pm line 848.
[23898] warn: Global symbol $type requires explicit package name at
/etc/mail/spamassassin/Botnet.pm line 848.
[23898] warn: Missing right curly or square bracket at
/etc/mail/spamassassin/Botnet.pm line 875, at end of line
[23898] warn: syntax error at /etc/mail/spamassassin/Botnet.pm line 875,
at EOF
[23898] warn: Compilation failed in require at
/usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PluginHandler.pm line 97.
Oops, yanked an entra line. Try this one instead:
--- Botnet.pm.orig 2007-06-15 16:47:33.0 -0500
+++ Botnet.pm 2007-06-15 17:35:33.0 -0500
@@ -703,7 +703,16 @@
($type =~ /^(?:A|MX)$/)
(defined $max)
($max =~ /^-?\d+$/) ) {
- $resolver = Net::DNS::Resolver-new();
+ $resolver = Net::DNS::Resolver-new(
+ udp_timeout = 3,
+ tcp_timeout = 3,
+ retrans = 0,
+ retry = 1,
+ persistent_tcp=0,
+ persistent_udp=0,
+ dnsrch=0,
+ defnames=0,
+ );
if ($query = $resolver-search($name, $type)) {
# found matches
$i = 0;
@@ -826,7 +835,16 @@
sub get_rdns {
my ($ip) = @_;
my ($query, @answer, $rr);
- my $resolver = Net::DNS::Resolver-new();
+ my $resolver = Net::DNS::Resolver-new(
+ udp_timeout = 3,
+ tcp_timeout = 3,
+ retrans = 0,
+ retry = 1,
+ persistent_tcp=0,
+ persistent_udp=0,
+ dnsrch=0,
+ defnames=0,
+ );
my $name = ;
if ($query = $resolver-query($ip, 'PTR', 'IN')) {
Beautiful, works like a charm!
Thanks Daniel!
Bill
RE: These are getting through SA...
What happens if Botnet is patched to use Mail::SpamAssassin::DnsResolver instead of Net::DNS::Resolver? I'm musuing about Net::DNS::Resolver's default timeouts and retries... Phil (probably barking up the wrong tree) -- Phil Randal Network Engineer Herefordshire Council Hereford, UK -Original Message- From: Bill Landry [mailto:[EMAIL PROTECTED] Sent: 12 June 2007 23:30 To: users@spamassassin.apache.org Subject: Re: These are getting through SA... Mark Martinec wrote the following on 6/12/2007 3:05 PM -0800: Bill, Mark, just curious if you are running Botnet? I found that some messages cause the Botnet RDNS test to timeout after hanging for about 30 seconds, and then network test randomly fail (primarily URIBL tests). I found that if I disable Botnet, then all network tests will run fine on the very same messages. Thanks, looks like the same cause (Botnet runs with Razor, dcc, etc., before the first and the second round of DNS launches). Please try the patch attached to http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511 (applies to SA 2.3.1 or 2.3.0), it is likely to fix these symptoms too. Mark Mark, I patched Dns.pm but this didn't resolve the issue for me. You can test with the sample messages I posted to bugzilla: http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5506 The only way I can get the URIBL tests to report hits it to run the messages through SA without the headers (samples without headers also posted to the bugzilla). Bill
Re: These are getting through SA...
Phil, What happens if Botnet is patched to use Mail::SpamAssassin::DnsResolver instead of Net::DNS::Resolver? I'm musuing about Net::DNS::Resolver's default timeouts and retries... Phil (probably barking up the wrong tree) It would do good if Botnet would impose a time limit on its DNS queries. It would also sidestep the Dns.pm problem, but not fix it. If the time spent by Razor+dcc+Botnet+(not sure what else) is longer than rbl_timeout, then replies to RBL queries are thrown away by mistake. Mark
Re: These are getting through SA...
Luis, I don't have any URIBL rules firing up (SA 3.2.0 from source here, most of the other relevant info is in the header of the mail I sent before to test). Where did you get them? [...] But the main difference between the live run and the ones I did with SA by itself (both as root and as user amavis) is the URIDNSBL hit. [...] From this debug, I see Amavis loading up the URIDNSBL plugin at startup, but lately it simply doesn't fire up on any spammy link (I googled for them, since the DDoS attack blocked the website). I came across the same issue yesterday, with the same type of a spam message, which would mostly get hits from URIBL tests, but lots of other RBL checks come back emptyhanded. On the first appearance it seems that SA under amavisd-new didn't fire on DNSBL tests, but spamassassin from a command line did. Investigating the problem more thoroughly turned out that even a command line SA check behaved intermittently, sometimes returning URIBL_BLACK, URIBL_JP_SURBL, etc, and sometimes none of these URIBL tests - they were timing out. What is your setting for rbl_timeout ? Mine was fairly low, 5 seconds, and I find the dynamic timeout (for rbl_timeout) cutback logic (man Mail::SpamAssassin::Conf) does not work as advertised: In addition, whenever the effective timeout is lowered due to addi- tional query results returning, the remaining queries are always given at least one more second before timing out Namely with 22 RBL results coming back, the last one (which was the crucial URIBL test) had a timeout of 0 and was ignored even though dns result did arrive. Moreover, there is a bug in Mail::SpamAssassin::Dns, where a late-spawned URIBL queries (which only start after Razor, DCC and Pyzor are run) are being timed against start time of the first wave of plain RBL dns queries, which are fired-off seconds earlier, so there is a good chance that URIBL queries time out in 0 seconds and their resultes are never collected. The problem is made worse when for example Razor itself also times out (thus extending time between the two rounds of dns queries being sent). Luis, check your DNS if it is responponding quickly, try extending rbl_timeout to maybe 10 seconds, see if there are many timeouts in RBL, URIBL, Razor or DCC queries. Mark
Re: These are getting through SA...
Well, I dint't have rbl_timeout set, but after your mail, I did. The DNSs I have set in resolv.conf are mine, they both cache and work as internal and external resolvers. But the UNLP NOC got screwed in the last days, so setting the timeout a little higher wont't hurt. Thanks for the suggestion. However, I upgraded to Amavis 2.5.1 yesterday (and rebuilt the AWL and the Bayes SQL databases, because they got corrupted) and everythig got back to normal. Updated several modules as Amavis required, and everything got back to the usual behavior. URIBL rules got fired (on several mails), and Razor and Pyzor got me results again. Additionally, SA stopped complaining about some minor issues when running sa-compile. Thanks again, Luix 2007/6/12, Mark Martinec [EMAIL PROTECTED]: Luis, I don't have any URIBL rules firing up (SA 3.2.0 from source here, most of the other relevant info is in the header of the mail I sent before to test). Where did you get them? [...] But the main difference between the live run and the ones I did with SA by itself (both as root and as user amavis) is the URIDNSBL hit. [...] From this debug, I see Amavis loading up the URIDNSBL plugin at startup, but lately it simply doesn't fire up on any spammy link (I googled for them, since the DDoS attack blocked the website). I came across the same issue yesterday, with the same type of a spam message, which would mostly get hits from URIBL tests, but lots of other RBL checks come back emptyhanded. On the first appearance it seems that SA under amavisd-new didn't fire on DNSBL tests, but spamassassin from a command line did. Investigating the problem more thoroughly turned out that even a command line SA check behaved intermittently, sometimes returning URIBL_BLACK, URIBL_JP_SURBL, etc, and sometimes none of these URIBL tests - they were timing out. What is your setting for rbl_timeout ? Mine was fairly low, 5 seconds, and I find the dynamic timeout (for rbl_timeout) cutback logic (man Mail::SpamAssassin::Conf) does not work as advertised: In addition, whenever the effective timeout is lowered due to addi- tional query results returning, the remaining queries are always given at least one more second before timing out Namely with 22 RBL results coming back, the last one (which was the crucial URIBL test) had a timeout of 0 and was ignored even though dns result did arrive. Moreover, there is a bug in Mail::SpamAssassin::Dns, where a late-spawned URIBL queries (which only start after Razor, DCC and Pyzor are run) are being timed against start time of the first wave of plain RBL dns queries, which are fired-off seconds earlier, so there is a good chance that URIBL queries time out in 0 seconds and their resultes are never collected. The problem is made worse when for example Razor itself also times out (thus extending time between the two rounds of dns queries being sent). Luis, check your DNS if it is responponding quickly, try extending rbl_timeout to maybe 10 seconds, see if there are many timeouts in RBL, URIBL, Razor or DCC queries. Mark -- - GNU-GPL: May The Source Be With You... Linux Registered User #448382. When I grow up, I wanna be like Theo... -
Re: These are getting through SA...
Luis, Namely with 22 RBL results coming back, the last one (which was the crucial URIBL test) had a timeout of 0 and was ignored even though dns result did arrive. Moreover, there is a bug in Mail::SpamAssassin::Dns, where a late-spawned URIBL queries (which only start after Razor, DCC and Pyzor are run) are being timed against start time of the first wave of plain RBL dns queries, which are fired-off seconds earlier, so there is a good chance that URIBL queries time out in 0 seconds and their resultes are never collected. I submitted a problem report, with a proposed patch: http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511 Things are much more predictable now. Mark
Re: These are getting through SA...
Mark Martinec wrote the following on 6/12/2007 3:53 AM -0800: Luis, I don't have any URIBL rules firing up (SA 3.2.0 from source here, most of the other relevant info is in the header of the mail I sent before to test). Where did you get them? [...] But the main difference between the live run and the ones I did with SA by itself (both as root and as user amavis) is the URIDNSBL hit. [...] From this debug, I see Amavis loading up the URIDNSBL plugin at startup, but lately it simply doesn't fire up on any spammy link (I googled for them, since the DDoS attack blocked the website). I came across the same issue yesterday, with the same type of a spam message, which would mostly get hits from URIBL tests, but lots of other RBL checks come back emptyhanded. On the first appearance it seems that SA under amavisd-new didn't fire on DNSBL tests, but spamassassin from a command line did. Investigating the problem more thoroughly turned out that even a command line SA check behaved intermittently, sometimes returning URIBL_BLACK, URIBL_JP_SURBL, etc, and sometimes none of these URIBL tests - they were timing out. What is your setting for rbl_timeout ? Mine was fairly low, 5 seconds, and I find the dynamic timeout (for rbl_timeout) cutback logic (man Mail::SpamAssassin::Conf) does not work as advertised: In addition, whenever the effective timeout is lowered due to addi- tional query results returning, the remaining queries are always given at least one more second before timing out Namely with 22 RBL results coming back, the last one (which was the crucial URIBL test) had a timeout of 0 and was ignored even though dns result did arrive. Moreover, there is a bug in Mail::SpamAssassin::Dns, where a late-spawned URIBL queries (which only start after Razor, DCC and Pyzor are run) are being timed against start time of the first wave of plain RBL dns queries, which are fired-off seconds earlier, so there is a good chance that URIBL queries time out in 0 seconds and their resultes are never collected. The problem is made worse when for example Razor itself also times out (thus extending time between the two rounds of dns queries being sent). Luis, check your DNS if it is responponding quickly, try extending rbl_timeout to maybe 10 seconds, see if there are many timeouts in RBL, URIBL, Razor or DCC queries. Mark Mark, just curious if you are running Botnet? I found that some messages cause the Botnet RDNS test to timeout after hanging for about 30 seconds, and then network test randomly fail (primarily URIBL tests). I found that if I disable Botnet, then all network tests will run fine on the very same messages. Bill
RE: These are getting through SA...
Well caught, Mark! I'd come to similar conclusions even without digging into the code when I saw DNS-related strangeness when I was testing SA 3.2.0 a few weeks back. I'll second your request that SA process all results it has collected on timeout, instead of discarding them. Cheers, Phil -Original Message- From: Mark Martinec [mailto:[EMAIL PROTECTED] Sent: 12 June 2007 17:20 To: users@spamassassin.apache.org Subject: Re: These are getting through SA... Luis, Namely with 22 RBL results coming back, the last one (which was the crucial URIBL test) had a timeout of 0 and was ignored even though dns result did arrive. Moreover, there is a bug in Mail::SpamAssassin::Dns, where a late-spawned URIBL queries (which only start after Razor, DCC and Pyzor are run) are being timed against start time of the first wave of plain RBL dns queries, which are fired-off seconds earlier, so there is a good chance that URIBL queries time out in 0 seconds and their resultes are never collected. I submitted a problem report, with a proposed patch: http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511 Things are much more predictable now. Mark
Re: These are getting through SA...
Bill, Mark, just curious if you are running Botnet? I found that some messages cause the Botnet RDNS test to timeout after hanging for about 30 seconds, and then network test randomly fail (primarily URIBL tests). I found that if I disable Botnet, then all network tests will run fine on the very same messages. Thanks, looks like the same cause (Botnet runs with Razor, dcc, etc., before the first and the second round of DNS launches). Please try the patch attached to http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511 (applies to SA 2.3.1 or 2.3.0), it is likely to fix these symptoms too. Mark
RE: These are getting through SA...
Bill, I was getting this sort of symptom without using Botnet. It's almost as if something's deadlocking somewhere in SA (until the timeout kicks in). Phil -Original Message- From: Bill Landry [mailto:[EMAIL PROTECTED] Sent: 12 June 2007 22:47 To: users@spamassassin.apache.org Subject: Re: These are getting through SA... Mark, just curious if you are running Botnet? I found that some messages cause the Botnet RDNS test to timeout after hanging for about 30 seconds, and then network test randomly fail (primarily URIBL tests). I found that if I disable Botnet, then all network tests will run fine on the very same messages. Bill
Re: These are getting through SA...
Mark Martinec wrote the following on 6/12/2007 3:05 PM -0800: Bill, Mark, just curious if you are running Botnet? I found that some messages cause the Botnet RDNS test to timeout after hanging for about 30 seconds, and then network test randomly fail (primarily URIBL tests). I found that if I disable Botnet, then all network tests will run fine on the very same messages. Thanks, looks like the same cause (Botnet runs with Razor, dcc, etc., before the first and the second round of DNS launches). Please try the patch attached to http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511 (applies to SA 2.3.1 or 2.3.0), it is likely to fix these symptoms too. Mark Mark, I patched Dns.pm but this didn't resolve the issue for me. You can test with the sample messages I posted to bugzilla: http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5506 The only way I can get the URIBL tests to report hits it to run the messages through SA without the headers (samples without headers also posted to the bugzilla). Bill
Re: These are getting through SA...
Bill, Mark, I patched Dns.pm but this didn't resolve the issue for me. You can test with the sample messages I posted to bugzilla: http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5506 Yes, it is the same problem as I describe in http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511 but to fix it requires my 'feature request' to be implemented: Now for the last part - a feature request. I think that no attempt has been made to collect already received DNS responses when timeout is reached. Given an asynchronous nature of DNS lookups in this module, I think it would be worthwhile to collect whatever is still in the IP receive queue after a timeout. The problem with BOTNET is that it tries to do a reverse DNS lookup on 66.17.235.109, which has broken DNS servers and none are reachable, so it hangs in sub get_rdns query($ip,'PTR','IN') for 24 seconds. After BOTNET finally times out, the Dns.pm harvest_dnsbl_queries tries to collect its RBL results, but abandons all attempts right away because it sees that 24 seconds has passed by and just declares them timed out, despite the fact that DNS results are waiting in TCP/IP received queue. Mark
Re: These are getting through SA...
Hi! They aren't scoring very much here... Return-Path: [EMAIL PROTECTED] X-Original-To: [EMAIL PROTECTED] Delivered-To: [EMAIL PROTECTED]@domain.com Received: from localhost (localhost [127.0.0.1]) by nahuel.biol.unlp.edu.ar (Postfix) with ESMTP id 660BE7B1FE; Fri, 8 Jun 2007 17:25:09 -0300 (ART) X-Prolocation-MailScanner-SpamCheck: spam, SpamAssassin (not cached, score=14.999, required 5, BAD_URI 1.50, CM_META_HK_NOW 2.50, CM_SPACED_DATE 0.50, PROLO_BLACK_DNSBL 3.00, RAZOR2_CF_RANGE_51_100 0.50, RAZOR2_CF_RANGE_E8_51_100 1.50, RAZOR2_CHECK 0.50, SPF_PASS -0.00, URIBL_JP_SURBL 4.00) Allmost 15, not bad i think. Bye, Raymond.
Re: These are getting through SA...
Luis Hernán Otegui schrieb: Hi, could somebody run this mail trough SA and give me the scores? They aren't scoring very much here... Hi, your mailing probably broke half of the email so these scores are only an estimate - if you want me to try again attach the mail as a raw text (or .eml as many clients call it) X-Spam-Report: * 0.0 MISSING_MID Missing Message-Id: header * 0.0 MISSING_DATE Missing Date: header * 2.5 MISSING_HB_SEP Missing blank line between message header and body * 0.0 DKIM_POLICY_SIGNSOME Domain Keys Identified Mail: policy says domain * signs some mails * 1.3 MISSING_HEADERS Missing To: header * 0.0 BAYES_50 BODY: Bayesian spam probability is 40 to 60% * [score: 0.5000] * 1.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level * above 50% * [cf: 100] * 0.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/) * 0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50% * [cf: 100] * 2.0 URIBL_BLACK Contains an URL listed in the URIBL blacklist * [URIs: teethcat.hk] * 1.8 MISSING_SUBJECT Missing Subject: header arni
Re: These are getting through SA...
Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from source here, most of the other relevant info is in the header of the mail I sent before to test). Where did you get them? Thanks, Luis 2007/6/8, Raymond Dijkxhoorn [EMAIL PROTECTED]: Hi! They aren't scoring very much here... Return-Path: [EMAIL PROTECTED] X-Original-To: [EMAIL PROTECTED] Delivered-To: [EMAIL PROTECTED]@domain.com Received: from localhost (localhost [127.0.0.1]) by nahuel.biol.unlp.edu.ar (Postfix) with ESMTP id 660BE7B1FE; Fri, 8 Jun 2007 17:25:09 -0300 (ART) X-Prolocation-MailScanner-SpamCheck: spam, SpamAssassin (not cached, score=14.999, required 5, BAD_URI 1.50, CM_META_HK_NOW 2.50, CM_SPACED_DATE 0.50, PROLO_BLACK_DNSBL 3.00, RAZOR2_CF_RANGE_51_100 0.50, RAZOR2_CF_RANGE_E8_51_100 1.50, RAZOR2_CHECK 0.50, SPF_PASS -0.00, URIBL_JP_SURBL 4.00) Allmost 15, not bad i think. Bye, Raymond. -- - GNU-GPL: May The Source Be With You... Linux Registered User #448382. When I grow up, I wanna be like Theo... -
Re: These are getting through SA...
Luis Hernán Otegui wrote: Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from source here, most of the other relevant info is in the header of the mail I sent before to test). Where did you get them? Run sa-update to get URIBL_BLACK and URIBL_GREY. Daryl
Re: These are getting through SA...
Hi! Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from source here, most of the other relevant info is in the header of the mail I sent before to test). Where did you get them? X-Prolocation-MailScanner-SpamCheck: spam, SpamAssassin (not cached, score=14.999, required 5, BAD_URI 1.50, CM_META_HK_NOW 2.50, CM_SPACED_DATE 0.50, PROLO_BLACK_DNSBL 3.00, RAZOR2_CF_RANGE_51_100 0.50, RAZOR2_CF_RANGE_E8_51_100 1.50, RAZOR2_CHECK 0.50, SPF_PASS -0.00, URIBL_JP_SURBL 4.00) Most likely since we get SURBL updates and so earlier then most of you do. Could be timings issues. Bye, Raymond.
Re: These are getting through SA...
Well, right now I'm running these commands to get updates: sa-update --gpgkey GPGKEY --channel saupdates.openprotect.com sa-update --gpgkey GPGKEY --channel updates.spamassassin.org sa-update doesn't download URIBL_BLACK and URIBL_GREY What am I doing wrong? Luis 2007/6/8, Daryl C. W. O'Shea [EMAIL PROTECTED]: Luis Hernán Otegui wrote: Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from source here, most of the other relevant info is in the header of the mail I sent before to test). Where did you get them? Run sa-update to get URIBL_BLACK and URIBL_GREY. Daryl -- - GNU-GPL: May The Source Be With You... Linux Registered User #448382. When I grow up, I wanna be like Theo... -
Re: These are getting through SA...
X-Spam-Status: No, score=3.964 tagged_above=-100 required=5
tests=[BAYES_99=3.5, HTML_30_40=0.463, HTML_MESSAGE=0.001]
To me, it looks like enough tokens were seen to flag it as BAYES_99, but
that the host and IP it came from didn't trigger any RBL hits, which
left your point score well under your 5.0 tolerance.
I can't speak for others, but I personally set my 'required' level to
3.5, so anything SpamAssassin flags as BAYES_99 automatically gets
flagged as spam. And since I use {SPAM _SCORE(0)_} as a subject rewrite,
I can catch any messages that score over another threshold (for me, it's
10.0) and filter them to /dev/null if they score that high -- the only
things that score that high are messages with bayes scores of 80-100
plus RBL hits.
Just my $0.02.
Re: These are getting through SA...
If you've got the current update from updates.spamassassin.org you've got a working set of rules for URIBL_BLACK and URIBL_GREY. It turns out that they didn't hit for Raymond either, so you won't see them in this case. Daryl Luis Hernán Otegui wrote: Well, right now I'm running these commands to get updates: sa-update --gpgkey GPGKEY --channel saupdates.openprotect.com sa-update --gpgkey GPGKEY --channel updates.spamassassin.org sa-update doesn't download URIBL_BLACK and URIBL_GREY What am I doing wrong? Luis 2007/6/8, Daryl C. W. O'Shea [EMAIL PROTECTED]: Luis Hernán Otegui wrote: Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from source here, most of the other relevant info is in the header of the mail I sent before to test). Where did you get them? Run sa-update to get URIBL_BLACK and URIBL_GREY. Daryl
Re: These are getting through SA...
OK, i?ve been googlin' around, and it seems like an issue between Amavis (or MailScanner, for waht I've found) and some unsupported versions of Net::DNS, because when I run the message through SA by itself, this comes out: Content analysis details: (9.7 points, 5.0 required) pts rule name description -- -- -1.8 ALL_TRUSTEDPassed through trusted hosts only via SMTP 3.5 BAYES_99 BODY: Bayesian spam probability is 99 to 100% [score: 0.] 0.0 MISSING_MIDMissing Message-Id: header 0.0 MISSING_DATE Missing Date: header 1.3 MISSING_HEADERSMissing To: header 2.0 URIBL_BLACKContains an URL listed in the URIBL blacklist [URIs: teethcat.hk] 1.8 MISSING_SUBJECTMissing Subject: header 2.5 FM_NO_FROM_OR_TO FM_NO_FROM_OR_TO 0.5 FM_NO_TO FM_NO_TO So I'm blaming it on Amavis... (Net::DNS 0.59 here)... I'll post this issue to the Amavis list. Luis 2007/6/8, Daryl C. W. O'Shea [EMAIL PROTECTED]: If you've got the current update from updates.spamassassin.org you've got a working set of rules for URIBL_BLACK and URIBL_GREY. It turns out that they didn't hit for Raymond either, so you won't see them in this case. Daryl Luis Hernán Otegui wrote: Well, right now I'm running these commands to get updates: sa-update --gpgkey GPGKEY --channel saupdates.openprotect.com sa-update --gpgkey GPGKEY --channel updates.spamassassin.org sa-update doesn't download URIBL_BLACK and URIBL_GREY What am I doing wrong? Luis 2007/6/8, Daryl C. W. O'Shea [EMAIL PROTECTED]: Luis Hernán Otegui wrote: Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from source here, most of the other relevant info is in the header of the mail I sent before to test). Where did you get them? Run sa-update to get URIBL_BLACK and URIBL_GREY. Daryl -- - GNU-GPL: May The Source Be With You... Linux Registered User #448382. When I grow up, I wanna be like Theo... -
Re: These are getting through SA...
Daryl C. W. O'Shea wrote the following on 6/8/2007 2:41 PM -0800: If you've got the current update from updates.spamassassin.org you've got a working set of rules for URIBL_BLACK and URIBL_GREY. It turns out that they didn't hit for Raymond either, so you won't see them in this case. Daryl Luis Hernán Otegui wrote: Well, right now I'm running these commands to get updates: sa-update --gpgkey GPGKEY --channel saupdates.openprotect.com sa-update --gpgkey GPGKEY --channel updates.spamassassin.org sa-update doesn't download URIBL_BLACK and URIBL_GREY What am I doing wrong? Luis 2007/6/8, Daryl C. W. O'Shea [EMAIL PROTECTED]: Luis Hernán Otegui wrote: Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from source here, most of the other relevant info is in the header of the mail I sent before to test). Where did you get them? Run sa-update to get URIBL_BLACK and URIBL_GREY. Daryl For what it's worth, and maybe it's different than the issues you are seeing, but I have been finding with SA 3.2.0, that some messages are not firing on URIBL tests. However, if the headers of the message are removed, the URIBL tests run fine. I posted a bug report to bugzilla (Bug 5506). I am running SA on Fedora 7 with perl 5.8.8 and was doing my test strictly with SA: spamassassin -t test.txt I did this because I wanted to take amavisd-new out of the picture. Also, someone mentioned Net::DNS, my version is 0.59. Bill
Re: These are getting through SA...
On Fri, 2007-06-08 at 18:46 -0300, Luis Hernán Otegui wrote:
OK, i?ve been googlin' around, and it seems like an issue between
Amavis (or MailScanner, for waht I've found) and some unsupported
versions of Net::DNS, because when I run the message through SA by
itself, this comes out:
Whatever you manually fed SA was even more borked than the inline
copy-n-paste of a message in your OP. Looking briefly at your original
paste, I do see these:
Date: Fri, 8 Jun 2007 20:25:53 -0100
From: Deana Adams [EMAIL PROTECTED]
Message-ID: [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Subject: Can you imagine that you are healthy?
However, your manual run hit hard on...
0.0 MISSING_MIDMissing Message-Id: header
0.0 MISSING_DATE Missing Date: header
1.3 MISSING_HEADERSMissing To: header
1.8 MISSING_SUBJECTMissing Subject: header
2.5 FM_NO_FROM_OR_TO FM_NO_FROM_OR_TO
0.5 FM_NO_TO FM_NO_TO
The -1.8 ALL_TRUSTED seems to support the assumption that you fed a
body only. Could be due to the exact details how you did it, though.
Also, this run didn't identify a HTML part at all...
The only difference that accounts for the spamminess in the second run
is the URIBL_BLACK hit. Maybe an oops, maybe a misconfiguration, maybe
due to not running in real time, but long after.
So I'm blaming it on Amavis... (Net::DNS 0.59 here)...
I don't see much evidence for this, yet. ;)
guenther
--
char *t=[EMAIL PROTECTED];
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: These are getting through SA...
What I copied and pasted into my message was the original spammy
message (the source of it) as IMP showed it. The posterior ALL_TRUSTED
occured because it has already been scanned and tagged by my servers.
But the main difference between the live run and the ones I did with
SA by itself (both as root and as user amavis) is the URIDNSBL hit.
Well, the blaming on Net::DNS wasn't an easy way out. I ran Amavis in
debug mode and spotted out some warnings about the use of (.) in
concatenation string in Util.pm (not literally, i'll post the correct
output on monday, when I get back to work). From this debug, I see
Amavis loading up the URIDNSBL plugin at startup, but lately it simply
doesn't fire up on any spammy link (I googled for them, since the DDoS
attack blocked the website).
Anyway, seems like my perl installation came out buggy (upgraded from
source to 5.8.8 before upgrading SA from 3.1.8 to 3.2.0), and it is
messing things up. Lately some errors with Net::SMTP came out when
reporting to SpamCop, so I guess I'll have to start it all over again
from scratch, but this time making sure all compiles ok.
Thanks,
Luis
2007/6/8, guenther [EMAIL PROTECTED]:
On Fri, 2007-06-08 at 18:46 -0300, Luis Hernán Otegui wrote:
OK, i?ve been googlin' around, and it seems like an issue between
Amavis (or MailScanner, for waht I've found) and some unsupported
versions of Net::DNS, because when I run the message through SA by
itself, this comes out:
Whatever you manually fed SA was even more borked than the inline
copy-n-paste of a message in your OP. Looking briefly at your original
paste, I do see these:
Date: Fri, 8 Jun 2007 20:25:53 -0100
From: Deana Adams [EMAIL PROTECTED]
Message-ID: [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Subject: Can you imagine that you are healthy?
However, your manual run hit hard on...
0.0 MISSING_MIDMissing Message-Id: header
0.0 MISSING_DATE Missing Date: header
1.3 MISSING_HEADERSMissing To: header
1.8 MISSING_SUBJECTMissing Subject: header
2.5 FM_NO_FROM_OR_TO FM_NO_FROM_OR_TO
0.5 FM_NO_TO FM_NO_TO
The -1.8 ALL_TRUSTED seems to support the assumption that you fed a
body only. Could be due to the exact details how you did it, though.
Also, this run didn't identify a HTML part at all...
The only difference that accounts for the spamminess in the second run
is the URIBL_BLACK hit. Maybe an oops, maybe a misconfiguration, maybe
due to not running in real time, but long after.
So I'm blaming it on Amavis... (Net::DNS 0.59 here)...
I don't see much evidence for this, yet. ;)
guenther
--
char *t=[EMAIL PROTECTED];
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
--
-
GNU-GPL: May The Source Be With You...
Linux Registered User #448382.
When I grow up, I wanna be like Theo...
-
Problem pr0n mail getting through SA
REMOVED_BY_THE_EXCHANGE_CONTENT_SCANNING_SERVICE_5CAE6CF4_4927_EThe original message content contained a virus or was blocked due to blocking rules and has been removed.
