DNSBL checks only on last untrusted host

2010-08-20 Thread Jacek Politowski
Recently, I've stumbled upon a situation, where my server rejected an
e-mail sent to me from blacklisted DSL via smarthost (beyond my
control).

Sender was from domain hosted on his MSP's server. His mail was sent
from blacklisted DSL via his MSP's smarthost (which of course requires
SMTP AUTH to relay from individual customers) to recipient on my
server.
This smarthost relayed e-mail to my server, where I run SpamAssassin
via Exim's DATA ACL (exiscan-acl).

Two of rules that hit (among others, but I'm wondering about those
two) were:
RCVD_IN_BL_SPAMCOP_NET
RCVD_IN_SORBS_WEB
and they were referencing original sender's DSL IP address, not
smarthost which delivered message to my server.

So, mail routing looked like this:
[remote_sender_sends_mail_from_blacklisted_DSL]--
[mail_is_accepted_by_his_MSP's_smarthost]--
[this_smarthost_relays_to_my_server_with_SA]--
[my_server_rejects_mail]


I'd like to convince my SpamAssassin only to do DNSBL checks on last
untrusted address (or addresses, if there are forged Received:
headers, impersonating my server) - that deliver directly to my
server, to avoid such situations in the future.


What is the preferred way to deal with situations like described
above?


-- 
Jacek Politowski


Re: DNSBL checks only on last untrusted host

2010-08-20 Thread Benny Pedersen

On fre 20 aug 2010 15:45:05 CEST, Jacek Politowski wrote

What is the preferred way to deal with situations like described
above?


trusted_networks smarthost-ip/cidr

here i exlude listed ips that are listed in some rbl, but clearly to  
me should not be listed, now you found such ip ? :)


--
xpoint http://www.unicom.com/pw/reply-to-harmful.html



Sought False Positives

2010-08-20 Thread Jan P. Kessler
 Hi,

we use spamassassin with the sought ruleset since several years at our
company. After the upgrade to from 3.2.5 to 3.3.1 we notice tons of
false-positives hitting on the rules JM_SOUGHT_1 and JM_SOUGHT_2.
Unfortunaley I can not give examples as these messages contain
confidental customer data (assurance company). We had more than 100
false-positives with these rules in the last 2 days.

I have drastically lowered the score from 4.0 to 1.0 for both rules and
wanted to ask if anybody else noticed that?

Cheers, Jan



Re: Sought False Positives

2010-08-20 Thread Karsten Bräckelmann
On Fri, 2010-08-20 at 17:12 +0200, Jan P. Kessler wrote:
 we use spamassassin with the sought ruleset since several years at our
 company. After the upgrade to from 3.2.5 to 3.3.1 we notice tons of

The SA upgrade is unrelated, the sought rules are the same for both and
frequently generated from recent spam. This is merely a timing
coincidence.

 false-positives hitting on the rules JM_SOUGHT_1 and JM_SOUGHT_2.
 Unfortunaley I can not give examples as these messages contain
 confidental customer data (assurance company). We had more than 100
 false-positives with these rules in the last 2 days.

I hope you can tell us the __SEEK_* sub-rules triggered, though. That
would help already. To extract these, either  (a) pipe such a message to
spamassassin -D, and get the sub-rule from the debug output, or  (b) add
a specific header only showing the sub-rules.

  spamassassin --cf=add_header all Subtests _SUBTESTS(,)_

Odds are, the FPs are some sort of stupid disclaimer that sneaked into
the spam corpus.

Once we know which sub-rule causes the FPs, and preferably get the full,
original string, we can add the sample to the ham corpus, preventing the
automated sought process from picking it up.

  guenther


-- 
char *t=\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}



Re: Sought False Positives

2010-08-20 Thread Karsten Bräckelmann
On Fri, 2010-08-20 at 17:47 +0200, Karsten Bräckelmann wrote:
 On Fri, 2010-08-20 at 17:12 +0200, Jan P. Kessler wrote:
  false-positives hitting on the rules JM_SOUGHT_1 and JM_SOUGHT_2.
  Unfortunaley I can not give examples as these messages contain
  confidental customer data (assurance company). We had more than 100
  false-positives with these rules in the last 2 days.
 
 I hope you can tell us the __SEEK_* sub-rules triggered, though. That
 would help already. To extract these, either  (a) pipe such a message to
 spamassassin -D, and get the sub-rule from the debug output, or  (b) add
 a specific header only showing the sub-rules.

A word of caution:  Do note that the seek sub-rules' names are generated
using a hash function, and thus identify the actual string matched!

You might want to check the string in 20_sought.cf, before disclosing
the seek ID. I'd be surprised if it contains sensitive data, tough --
after all, it is found massively in spam.


   spamassassin --cf=add_header all Subtests _SUBTESTS(,)_
 
 Odds are, the FPs are some sort of stupid disclaimer that sneaked into
 the spam corpus.
 
 Once we know which sub-rule causes the FPs, and preferably get the full,
 original string, we can add the sample to the ham corpus, preventing the
 automated sought process from picking it up.

-- 
char *t=\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}



Re: Sought False Positives

2010-08-20 Thread Rob McEwen
I think the problem is the following rule in sought:

body __SEEK_2TRLES  /Facebook, Inc\. P\.O\. Box 10005, Palo Alto, CA 94303/

which is currently hitting on many (or maybe even all ALL?) legitimate
facebook notifications (along with the ones generated by spammers)

-- 
Rob McEwen
http://dnsbl.invaluement.com/
r...@invaluement.com
+1 (478) 475-9032




Re: DNSBL checks only on last untrusted host

2010-08-20 Thread Jacek Politowski
On Fri, Aug 20, 2010 at 04:11:34PM +0200, Benny Pedersen wrote:

trusted_networks smarthost-ip/cidr

here i exlude listed ips that are listed in some rbl, but clearly to
me should not be listed, now you found such ip ? :)

Actually, the IP I've found _should_ be listed in DNSBL - I don't want
to receive any e-mail directly from this host (some DSL line with
abusable web server running on it...).

Receiving e-mails via some_big_MSP_smarthost is completely another
thing.

But I don't want to constantly monitor and add to whitelisting rules
all the smarthosts that might be sending e-mails to us -- there are
far too many of them.

I'd really like limit SpamAssassin's RCVD_* DNSBL checks only to
hosts that directly deliver e-mails to our servers, but it seems I'm
missing something in SA documentation (I can hardly believe there is
no such possibility in SA).


-- 
Jacek Politowski


Re: Sought False Positives

2010-08-20 Thread John Hardin

On Fri, 20 Aug 2010, Karsten Br�ckelmann wrote:


On Fri, 2010-08-20 at 17:47 +0200, Karsten Bräckelmann wrote:

On Fri, 2010-08-20 at 17:12 +0200, Jan P. Kessler wrote:
false-positives hitting on the rules JM_SOUGHT_1 and JM_SOUGHT_2. 
Unfortunaley I can not give examples as these messages contain 
confidental customer data (assurance company). We had more than 100 
false-positives with these rules in the last 2 days.


I hope you can tell us the __SEEK_* sub-rules triggered, though. That 
would help already. To extract these, either (a) pipe such a message to 
spamassassin -D, and get the sub-rule from the debug output, or (b) add 
a specific header only showing the sub-rules.


A word of caution:  Do note that the seek sub-rules' names are generated 
using a hash function, and thus identify the actual string matched!


You might want to check the string in 20_sought.cf, before disclosing
the seek ID. I'd be surprised if it contains sensitive data, tough --
after all, it is found massively in spam.


...as well as in SA SVN. The matches can't be confidential as they're 
generated from public sources. The non-matching bits are what is 
confidential.


I agree with Karsten, it's most likely disclaimer text that doesn't have a 
ham exclusion in the SOUGHT rule generator.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  I'll have that son of a bitch eating out of dumpsters in less than
  two years.   -- MS CEO Steve Ballmer, on RedHat CEO Matt Szulik
---
 4 days until the 1931st anniversary of the destruction of Pompeii

Re: DNSBL checks only on last untrusted host

2010-08-20 Thread John Hardin

On Fri, 20 Aug 2010, Jacek Politowski wrote:


Actually, the IP I've found _should_ be listed in DNSBL - I don't want
to receive any e-mail directly from this host (some DSL line with
abusable web server running on it...).

Receiving e-mails via some_big_MSP_smarthost is completely another
thing.


It sounds like you've somehow defined the MSP smarthost as either trusted 
or internal.


Take a look at its public IP address and your definitions of trusted and 
internal hosts.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  I'll have that son of a bitch eating out of dumpsters in less than
  two years.   -- MS CEO Steve Ballmer, on RedHat CEO Matt Szulik
---
 4 days until the 1931st anniversary of the destruction of Pompeii


Re: DNSBL checks only on last untrusted host

2010-08-20 Thread Karsten Bräckelmann
On Fri, 2010-08-20 at 20:34 +0200, Jacek Politowski wrote:
 Actually, the IP I've found _should_ be listed in DNSBL - I don't want
 to receive any e-mail directly from this host (some DSL line with
 abusable web server running on it...).
 
 Receiving e-mails via some_big_MSP_smarthost is completely another
 thing.

 I'd really like limit SpamAssassin's RCVD_* DNSBL checks only to
 hosts that directly deliver e-mails to our servers, but it seems I'm
 missing something in SA documentation (I can hardly believe there is
 no such possibility in SA).

Well, there is no single option to limit all such DNSBL tests to the
handing-over host. Whether the lookup will be limited to the last
external hop, or if all external hosts are tested for is handled on a
case-by-case basis in the eval() rule's definition.

Because it depends. Some lists are suitable for deep-parsing. Some are
not.


Moreover, IMHO you are barking up the wrong tree. In your OP you said, a
message has been *rejected* by your SMTP. Yet, you are focusing entirely
on the RCVD_IN_BL_SPAMCOP_NET and RCVD_IN_SORBS_WEB hits. Which by
itself won't even push the score above the default spam threshold.

Thus, very vital but left out parts to the puzzle are,  (a) which rules
triggered in addition to them, and  (b) at what threshold does your SMTP
reject a message?

The combined score of these rules is no where even close to a sensible
rejection limit. Whatever else the message tripped on, it accounts for
the lions-share.


-- 
char *t=\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}



Re: DNSBL checks only on last untrusted host

2010-08-20 Thread Karsten Bräckelmann
On Fri, 2010-08-20 at 20:54 +0200, Karsten Bräckelmann wrote:
 Because it depends. Some lists are suitable for deep-parsing. Some are
 not.
 
 
 Moreover, IMHO you are barking up the wrong tree. In your OP you said, a
 message has been *rejected* by your SMTP. Yet, you are focusing entirely
 on the RCVD_IN_BL_SPAMCOP_NET and RCVD_IN_SORBS_WEB hits. Which by
 itself won't even push the score above the default spam threshold.
 
 Thus, very vital but left out parts to the puzzle are,  (a) which rules
 triggered in addition to them, and  (b) at what threshold does your SMTP
 reject a message?
 
 The combined score of these rules is no where even close to a sensible
 rejection limit. Whatever else the message tripped on, it accounts for
 the lions-share.

Just to back up my claim with numbers, here are the scores for both 3.2
and 3.3 branches. Minimally edited for readability.

  $ egrep 'RCVD_IN_(BL_SPAMCOP_NET|SORBS_WEB)' 3.[23]/rules/50_scores.cf

  3.2/rules/50_scores.cf: score RCVD_IN_BL_SPAMCOP_NET 0 2.188 0 1.960
  3.2/rules/50_scores.cf: score RCVD_IN_SORBS_WEB  0 1.117 0 0.619

  3.3/rules/50_scores.cf: score RCVD_IN_BL_SPAMCOP_NET 0 1.246 0 1.347
  3.3/rules/50_scores.cf: score RCVD_IN_SORBS_WEB  0 0.614 0 0.770

As you can see, even the aging 3.2 rule-set with Bayes disabled scores
these at ~3.3 -- the worst possible combination, and yet still some way
to go to cross the spam threshold of 5.0. Enabling Bayes, or using the
latest stable SA release, only increases the buffer to be considered
spammy.

These numbers have been optimized with a spam threshold of 5.0 (at the
time of their creation) -- to *minimize* false classification [1], while
considering FPs more severe than FNs.

With that in mind, a sensible reject limit is 8.0 or even higher [2]. In
which case the remaining hits account for 4.7 -- or in other words,
(almost) would have pushed the message in question over the spam
threshold, even without those RCVD_IN_* hits.


[1] It is impossible to eliminate FPs and FNs at the same time. In your
OP you mentioned a single rejected message, right? ;)

[2] Based on experience and years of discussion on this list.

-- 
char *t=\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}



Re: DNSBL checks only on last untrusted host

2010-08-20 Thread Alexandre Chapellon
Le vendredi 20 août 2010 à 20:34 +0200, Jacek Politowski a écrit :

 On Fri, Aug 20, 2010 at 04:11:34PM +0200, Benny Pedersen wrote:
 
 trusted_networks smarthost-ip/cidr
 
 here i exlude listed ips that are listed in some rbl, but clearly to
 me should not be listed, now you found such ip ? :)
 
 Actually, the IP I've found _should_ be listed in DNSBL - I don't want
 to receive any e-mail directly from this host (some DSL line with
 abusable web server running on it...).
 
 Receiving e-mails via some_big_MSP_smarthost is completely another
 thing.
 
 But I don't want to constantly monitor and add to whitelisting rules
 all the smarthosts that might be sending e-mails to us -- there are
 far too many of them.
 
 I'd really like limit SpamAssassin's RCVD_* DNSBL checks only to
 hosts that directly deliver e-mails to our servers, but it seems I'm
 missing something in SA documentation (I can hardly believe there is
 no such possibility in SA).

IMHO having SA doing this is only a valid choice  if your receiving
hosts (MX) can't do it by itself at SMTP time.
If your want to reject email sent directly by DSL/dialup/ or any dyn
client... use RBL at connect time and reject mail if client is in the
list.

Spamhaus PBL in designed that way and cbl.abuseat.org can be used too
for that (lists hijacked/zombies machines if I remmeber well) thoose
lists (at list pbl) should not any MSP smarthost contains.
Alternatively, you can have rules thats reject mails at smtp time based
on the reversed dnsstring of client (reject if looks like
'4.3.2.1-some.dyn.dsl.-isp.tld')

This is much less ressource consuming than any parsing. And you can then
keep using deep parsing (with great caution about the lists you
uses) to help score spam.

I really think the big part of this work do not belongs to spamassassin
but to the MTA on the MX.

Regards

 
 




Re: Sought False Positives

2010-08-20 Thread Emin Akbulut
Training SA instead of debugging is much easier sometime,
I did give up with errors and do my own workarounds;
I made _spam and _ham dirs in SA dir and fill them with
spam or ham messages when I find a few, then fire the script:


@REM Train Spamassassin
c:
cd \NET\SpamAssassinWin32-EX

@REM Learn spam
sa-learn.exe --spam _spam
del _spam\*.* /q

@REM Learn ham
sa-learn.exe --ham _ham
@REM Resend false positives
copy _ham\*.* C:\inetpub\mailroot\Pickup
del _ham\*.* /q

@pause



Now I got the best statistics...



On Fri, Aug 20, 2010 at 9:44 PM, John Hardin jhar...@impsec.org wrote:

 On Fri, 20 Aug 2010, Karsten Bräckelmann wrote:

  On Fri, 2010-08-20 at 17:47 +0200, Karsten Bräckelmann wrote:

 On Fri, 2010-08-20 at 17:12 +0200, Jan P. Kessler wrote:

 false-positives hitting on the rules JM_SOUGHT_1 and JM_SOUGHT_2.
 Unfortunaley I can not give examples as these messages contain confidental
 customer data (assurance company). We had more than 100 false-positives 
 with
 these rules in the last 2 days.




Re: Sought False Positives

2010-08-20 Thread Benny Pedersen

On fre 20 aug 2010 19:42:04 CEST, Rob McEwen wrote


body __SEEK_2TRLES  /Facebook, Inc\. P\.O\. Box 10005, Palo Alto, CA 94303/

which is currently hitting on many (or maybe even all ALL?) legitimate
facebook notifications (along with the ones generated by spammers)


dkim signed ?, spf passed ?

--
xpoint http://www.unicom.com/pw/reply-to-harmful.html



Re: DNSBL checks only on last untrusted host

2010-08-20 Thread Benny Pedersen

On fre 20 aug 2010 20:34:51 CEST, Jacek Politowski wrote

I'd really like limit SpamAssassin's RCVD_* DNSBL checks only to
hosts that directly deliver e-mails to our servers, but it seems I'm
missing something in SA documentation (I can hardly believe there is
no such possibility in SA).


trusted_networks smarthost/cidr
blacklist_from spammers-domain

end result better ? :)

trusted_networks neotralize ip blacklist that include whitelisted ips

but blacklist_from does not care about ip

--
xpoint http://www.unicom.com/pw/reply-to-harmful.html



Re: DNSBL checks only on last untrusted host

2010-08-20 Thread Daniel J McDonald
On Fri, 2010-08-20 at 20:34 +0200, Jacek Politowski wrote:
 On Fri, Aug 20, 2010 at 04:11:34PM +0200, Benny Pedersen wrote:
 
 I'd really like limit SpamAssassin's RCVD_* DNSBL checks only to
 hosts that directly deliver e-mails to our servers, but it seems I'm
 missing something in SA documentation (I can hardly believe there is
 no such possibility in SA).

change: 
header RCVD_IN_BL_SPAMCOP_NET eval:check_rbl_txt('spamcop',
'bl.spamcop.net.', '(?i:spamcop)')
to:
header RCVD_IN_BL_SPAMCOP_NET eval:check_rbl_txt('spamcop-lastexternal',
'bl.spamcop.net.', '(?i:spamcop)')



-- 
Daniel J McDonald, CCIE # 2495, CISSP # 78281, CNX
www.austinenergy.com


Re: DNSBL checks only on last untrusted host

2010-08-20 Thread Jacek Politowski
On Fri, Aug 20, 2010 at 08:54:57PM +0200, Karsten Bräckelmann wrote:
On Fri, 2010-08-20 at 20:34 +0200, Jacek Politowski wrote:

 I'd really like limit SpamAssassin's RCVD_* DNSBL checks only to
 hosts that directly deliver e-mails to our servers, but it seems I'm
 missing something in SA documentation (I can hardly believe there is
 no such possibility in SA).

Well, there is no single option to limit all such DNSBL tests to the
handing-over host. Whether the lookup will be limited to the last
external hop, or if all external hosts are tested for is handled on a
case-by-case basis in the eval() rule's definition.

Moreover, IMHO you are barking up the wrong tree. In your OP you said, a
message has been *rejected* by your SMTP. Yet, you are focusing entirely
on the RCVD_IN_BL_SPAMCOP_NET and RCVD_IN_SORBS_WEB hits. Which by
itself won't even push the score above the default spam threshold.

Unfortunately, number of spam getting through, while I was using
default SpamAssassin configuration, was way too high. So I'm playing a
bit with a razor here (hoping I won't hurt myself too much).


I've made some statistics, which showed that most of spams getting
through scored (almost) only on a few DNSBL rules, so I raised the
score for them (but still not high enough to block e-mail mail with
single DNSBL hit).

This, however, left me with the situation I described in my first post.


I was hoping I'll be able to limit depth of Received: checks in
SA. This seemed like an easier option than implementing such logic
directly in the MTA, as most of required stuff is already present in
SpamAssassin.

I don't think I can afford rejecting emails based solely on just one
DNSBL - I don't trust any of them that much.

So, probably I'll just have to write my own checks for SA, giving them
scores useful in my situation.


-- 
Jacek Politowski


Re: DNSBL checks only on last untrusted host

2010-08-20 Thread Jacek Politowski
On Fri, Aug 20, 2010 at 03:55:13PM -0500, Daniel J McDonald wrote:
On Fri, 2010-08-20 at 20:34 +0200, Jacek Politowski wrote:

 I'd really like limit SpamAssassin's RCVD_* DNSBL checks only to
 hosts that directly deliver e-mails to our servers,

change: 
header RCVD_IN_BL_SPAMCOP_NET eval:check_rbl_txt('spamcop',
'bl.spamcop.net.', '(?i:spamcop)')
to:
header RCVD_IN_BL_SPAMCOP_NET eval:check_rbl_txt('spamcop-lastexternal',
'bl.spamcop.net.', '(?i:spamcop)')

It seems like the answer I was looking for. Thank you!


-- 
Jacek Politowski


Re: Sought False Positives

2010-08-20 Thread Rob McEwen
Benny Pedersen wrote:
 On fre 20 aug 2010 19:42:04 CEST, Rob McEwen wrote
 body __SEEK_2TRLES  /Facebook, Inc\. P\.O\. Box 10005, Palo Alto, CA
 94303/

 which is currently hitting on many (or maybe even all ALL?) legitimate
 facebook notifications (along with the ones generated by spammers)
 dkim signed ?, spf passed ? 

Yes and yes. But need I even check when I've already confirmed that they
were sent from IPs assigned to facebook.com by ARIN?

-- 
Rob McEwen
http://dnsbl.invaluement.com/
r...@invaluement.com
+1 (478) 475-9032




Re: Sought False Positives

2010-08-20 Thread Benny Pedersen

On fre 20 aug 2010 23:09:15 CEST, Rob McEwen wrote

Yes and yes. But need I even check when I've already confirmed that they
were sent from IPs assigned to facebook.com by ARIN?


test dkim, all you need to know it is from facebook

here if its facebookapp.com dkim signed i skip spf testing, easy to do  
in amavisd-new, thanks Mark


--
xpoint http://www.unicom.com/pw/reply-to-harmful.html



Re: DNSBL checks only on last untrusted host

2010-08-20 Thread Ned Slider

On 20/08/10 22:08, Jacek Politowski wrote:

On Fri, Aug 20, 2010 at 03:55:13PM -0500, Daniel J McDonald wrote:

On Fri, 2010-08-20 at 20:34 +0200, Jacek Politowski wrote:



I'd really like limit SpamAssassin's RCVD_* DNSBL checks only to
hosts that directly deliver e-mails to our servers,



change:
header RCVD_IN_BL_SPAMCOP_NET eval:check_rbl_txt('spamcop',
'bl.spamcop.net.', '(?i:spamcop)')
to:
header RCVD_IN_BL_SPAMCOP_NET eval:check_rbl_txt('spamcop-lastexternal',
'bl.spamcop.net.', '(?i:spamcop)')


It seems like the answer I was looking for. Thank you!




Make sure you make that change to your local.cf (or similar), and not in 
the original 
/var/lib/spamassassin/version/updates_spamassassin_org/20_dnsbl_tests.cf 
file otherwise sa-update will overwrite it the first time a set of 
updates come down the pipe.




Re: DNSBL checks only on last untrusted host

2010-08-20 Thread Karsten Bräckelmann
On Fri, 2010-08-20 at 23:05 +0200, Jacek Politowski wrote:
 On Fri, Aug 20, 2010 at 08:54:57PM +0200, Karsten Bräckelmann wrote:
  Moreover, IMHO you are barking up the wrong tree. In your OP you said, a
  message has been *rejected* by your SMTP. Yet, you are focusing entirely
  on the RCVD_IN_BL_SPAMCOP_NET and RCVD_IN_SORBS_WEB hits. Which by
  itself won't even push the score above the default spam threshold.
 
 Unfortunately, number of spam getting through, while I was using
 default SpamAssassin configuration, was way too high. So I'm playing a
 bit with a razor here (hoping I won't hurt myself too much).

Ah! A self-inflicted wound. :)


 I've made some statistics, which showed that most of spams getting
 through scored (almost) only on a few DNSBL rules, so I raised the
 score for them (but still not high enough to block e-mail mail with
 single DNSBL hit).
 
 This, however, left me with the situation I described in my first post.

Good to see you didn't raise any of them to a poison-pill single-hit
kill. Preserving the fundamental scoring approach of SA.

However, it appears you raised the score of some rules too much. Way too
much. The stock scores are set for a reason. And btw, you still didn't
tell your local scores and reject threshold.

If the $version stock doesn't cut it for you any more, I'd say upgrading
SA to the latest version is most likely to help -- seriously help.


 I was hoping I'll be able to limit depth of Received: checks in

As I said -- you can, but that's on a per-rule basis. The hop(s) checked
against any given BL are carefully considered in stock SA.

Anyway, the reason you want to do this in the first place is your
messing with the scores. A band-aid, to fix side-effects of excessive
score raising. You just entered whack-a-mole level.


 SA. This seemed like an easier option than implementing such logic
 directly in the MTA, as most of required stuff is already present in
 SpamAssassin.
 
 I don't think I can afford rejecting emails based solely on just one
 DNSBL - I don't trust any of them that much.
 
 So, probably I'll just have to write my own checks for SA, giving them
 scores useful in my situation.

That definitely sounds like a good idea. :)  Better than what I just
understood you have done before.


-- 
char *t=\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}



Re: DNSBL checks only on last untrusted host

2010-08-20 Thread Karsten Bräckelmann
   I'd really like limit SpamAssassin's RCVD_* DNSBL checks only to
   hosts that directly deliver e-mails to our servers,
 
 change: 
 header RCVD_IN_BL_SPAMCOP_NET eval:check_rbl_txt('spamcop',
 'bl.spamcop.net.', '(?i:spamcop)')

 It seems like the answer I was looking for. Thank you!

True. As I've hinted at before.

However, while this indeed is the answer you've been looking for, it
won't solve your problem. See my previous post, just a few minutes ago.

The only reason, why you asked about this, and why it will help in that
*one* case you encountered is, because it works around the very specific
situation of that *one* instance.

It will not generally solve the underlying issue of countering your
local score raising. Again, see my previous post.

You have entered a game of whack-a-mole. You will see more and more of
these as time and mail goes by. More band-aid won't help healing your
wounds.


-- 
char *t=\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}



Re: DNSBL checks only on last untrusted host

2010-08-20 Thread Benny Pedersen

On fre 20 aug 2010 23:33:09 CEST, Ned Slider wrote


Make sure you make that change to your local.cf (or similar), and not


or more generic make a ticket if its in public intrest to make it  
generic change :)


--
xpoint http://www.unicom.com/pw/reply-to-harmful.html



Re: DNSBL checks only on last untrusted host

2010-08-20 Thread Karsten Bräckelmann
On Fri, 2010-08-20 at 23:58 +0200, Benny Pedersen wrote:
 or more generic make a ticket if its in public intrest to make it  
 generic change :)

It is not.  Did you follow the entire thread?


-- 
char *t=\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}



Re: DNSBL checks only on last untrusted host

2010-08-20 Thread Benny Pedersen

On lør 21 aug 2010 00:15:48 CEST, Karsten Bräckelmann wrote


On Fri, 2010-08-20 at 23:58 +0200, Benny Pedersen wrote:

or more generic make a ticket if its in public intrest to make it
generic change :)


It is not.  Did you follow the entire thread?


i hope i have, i have a night work to change my server move from a  
nearly dead system disk over to another server, hope its will stay  
with me now, kernel unrecoverly errors in ext3 is not fun :/


sorry to be ot here, will stop from now and get my work done

--
xpoint http://www.unicom.com/pw/reply-to-harmful.html



Re: DNSBL checks only on last untrusted host

2010-08-20 Thread Karsten Bräckelmann
On Sat, 2010-08-21 at 00:23 +0200, Benny Pedersen wrote:
   or more generic make a ticket if its in public intrest to make it
   generic change :)
 
  It is not.  Did you follow the entire thread?

 sorry to be ot here, will stop from now and get my work done

Don't worry, it wasn't off-topic. It just is not a case that applies to
the general user. So, no bug report about this. ;)

FWIW, I never have, nor will call OT threads to an end without a good
reason, as long as they are at least related to SA. Don't get that
impression.


-- 
char *t=\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}



Whitelist question

2010-08-20 Thread Alex
Hi,

I'm trying to use whitelist_from_rcvd and it doesn't appear to be
working. I'm trying to whitelist mail from the AZ lottery. Here are
the headers from the email:

Received: from AZMTAQS01.AZ.GOV (azmtaprd01.az.gov [159.87.126.8])
From: Arizona Lottery email.re...@azlottery.gov

Isn't this the proper way to use this?

whitelist_from_rcvd email.re...@azlottery.gov az.gov

What am I doing wrong?

Thanks,
Alex


Re: Whitelist question

2010-08-20 Thread Matt Kettler

 On 8/20/2010 9:09 PM, Alex wrote:

Hi,

I'm trying to use whitelist_from_rcvd and it doesn't appear to be
working. I'm trying to whitelist mail from the AZ lottery. Here are
the headers from the email:

Received: from AZMTAQS01.AZ.GOV (azmtaprd01.az.gov [159.87.126.8])
From: Arizona Lotteryemail.re...@azlottery.gov

Isn't this the proper way to use this?

whitelist_from_rcvd email.re...@azlottery.gov az.gov

What am I doing wrong?


That should work. Assuming that:

1) there is a by clause you cut off that Received: header.
2) The host that is receiving the mail from az.gov is trusted by SA.
3) az.gov is *NOT* trusted by SA.

For 2 and 3 you might want to run a copy of the message through 
spamassassin -D and see what the list of trusted and untrusted relays are.




Re: Whitelist question

2010-08-20 Thread Henrik K
On Fri, Aug 20, 2010 at 11:39:05PM -0400, Matt Kettler wrote:
  On 8/20/2010 9:09 PM, Alex wrote:
 Hi,
 
 I'm trying to use whitelist_from_rcvd and it doesn't appear to be
 working. I'm trying to whitelist mail from the AZ lottery. Here are
 the headers from the email:
 
 Received: from AZMTAQS01.AZ.GOV (azmtaprd01.az.gov [159.87.126.8])
 From: Arizona Lotteryemail.re...@azlottery.gov
 
 Isn't this the proper way to use this?
 
 whitelist_from_rcvd email.re...@azlottery.gov az.gov
 
 What am I doing wrong?
 
 That should work. Assuming that:
 
 1) there is a by clause you cut off that Received: header.
 2) The host that is receiving the mail from az.gov is trusted by SA.
 3) az.gov is *NOT* trusted by SA.
 
 For 2 and 3 you might want to run a copy of the message through
 spamassassin -D and see what the list of trusted and untrusted
 relays are.

You need to use _envelope_ sender (e.g. Return-Path), not From.



Re: Whitelist question

2010-08-20 Thread Henrik K
On Sat, Aug 21, 2010 at 08:16:58AM +0300, Henrik K wrote:
 On Fri, Aug 20, 2010 at 11:39:05PM -0400, Matt Kettler wrote:
   On 8/20/2010 9:09 PM, Alex wrote:
  Hi,
  
  I'm trying to use whitelist_from_rcvd and it doesn't appear to be
  working. I'm trying to whitelist mail from the AZ lottery. Here are
  the headers from the email:
  
  Received: from AZMTAQS01.AZ.GOV (azmtaprd01.az.gov [159.87.126.8])
  From: Arizona Lotteryemail.re...@azlottery.gov
  
  Isn't this the proper way to use this?
  
  whitelist_from_rcvd email.re...@azlottery.gov az.gov
  
  What am I doing wrong?
  
  That should work. Assuming that:
  
  1) there is a by clause you cut off that Received: header.
  2) The host that is receiving the mail from az.gov is trusted by SA.
  3) az.gov is *NOT* trusted by SA.
  
  For 2 and 3 you might want to run a copy of the message through
  spamassassin -D and see what the list of trusted and untrusted
  relays are.
 
 You need to use _envelope_ sender (e.g. Return-Path), not From.

Never mind, I was confusing it with spf and read the docs..