Re: DNSWL overriding bayes_99 and bayes_999 rules

[Greg Troxel]

Steve Dondley  writes:

> Note: I've changed the score of RCVD_IN_DNSWL_HI hits to -2.0 from
> -5.0 until I get my misconfiguration figured out. Thanks for your
> patience.

Fair enough; that's not an unreasonable thing to do.

Probably you want to turn report_safe to 0 for doing this testing.


> Content analysis details:   (23.2 points, 5.0 required)

I would expect your MTA to be configured to hard reject mail that has a
score of 23.  15 if you're cautious, 10 if you're aggressive.


>  pts rule name  description
>  -- 
> --
> -2.0 RCVD_IN_DNSWL_HI   RBL: Sender listed at
> https://www.dnswl.org/,
> high trust
> [203.160.71.180 listed in list.dnswl.org]
I looked up this, and the other one, and didn't find them in dnswl.   As
others said, if you are using public DNS, stop doing that immediately.
And, run the dnswl queries with dig or host yourself on your own machine.

> -0.0 RCVD_IN_MSPIKE_H2  RBL: Average reputation (+2)
> [203.160.71.180 listed in wl.mailspike.net]

This is H2, not higher, which is consistent with DNSWL_LO or
DNSWL_NONE.  (Just a comment.)

>  2.7 RCVD_IN_PSBL   RBL: Received via a relay in PSBL
> [203.160.71.180 listed in psbl.surriel.com]
>  3.5 BAYES_99   BODY: Bayes spam probability is 99 to 100%
> [score: 1.]
>  0.5 BAYES_999  BODY: Bayes spam probability is 99.9 to 100%
> [score: 1.]
>  2.0 LOCAL_SPAM_TLD Domain originates a lot of spam
>  1.0 LOCAL_UNCOMMON_TLD From address is not a common TLD
>  1.3 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in
> bl.spamcop.net
>  [Blocked - see
> ]
>  1.3 RCVD_IN_VALIDITY_RPBL  RBL: Relay in Validity RPBL,
> https://senderscore.org/blocklistlookup/
>[203.160.71.180 listed in
> bl.score.senderscore.com]

So the address is in some blocklists.

> Received-SPF: Softfail (mailfrom) identity=mailfrom;
> client-ip=203.160.71.180; helo=yahoo.co.jp;
> envelope-from=qy5cbma-yu...@yahoo.co.jp; receiver=
> Received: from yahoo.co.jp (unknown [203.160.71.180])
>   by email.dondley.com (Postfix) with SMTP id 842C2210C0
>   for ; Sat, 10 Apr 2021 05:49:55 -0400 (EDT)

Note the lack of rDNS, and what is probably a spoofed HELO.


So overall SA di the right thing: 23.5 is a score for an email that is
so spammy that I have no qualms about outright rejecting it.


signature.asc
Description: PGP signature

Re: Spamassassin reporting IP address is whitelisted by DNSWL.org but DNSWL.org reports it is not

[Steve Dondley]

On 2021-04-10 03:20 PM, Bill Cole wrote:

On 10 Apr 2021, at 14:53, Steve Dondley wrote:

I'm very, very sorry to beat a dead horse, but I'm deeply confused by 
the "RCVD_IN_DNSWL_HI" rule which appears to be reporting incorrectly 
on my system.


STOP USING ANY PUBLIC DNS RESOLVERS WITH ANY MAIL SERVERS!


For the record, my nameserver setting in /etc/resolv.conf was some local 
IP address which presumably used an Amazon Web Service (AWS) DNS server.


After changing the IP address to 127.0.0.1 in that file, it changed 
itself back to the original IP address after some short period of time. 
To fix this, follow the appropriate instructions here: 
https://aws.amazon.com/premiumsupport/knowledge-center/ec2-static-dns-ubuntu-debian/

Re: Spamassassin reporting IP address is whitelisted by DNSWL.org but DNSWL.org reports it is not

[Bill Cole]

On 10 Apr 2021, at 14:53, Steve Dondley wrote:

I'm very, very sorry to beat a dead horse, but I'm deeply confused by 
the "RCVD_IN_DNSWL_HI" rule which appears to be reporting incorrectly 
on my system.


STOP USING ANY PUBLIC DNS RESOLVERS WITH ANY MAIL SERVERS!

Some of these will return bogus values instead of a proper NXDOMAIN, 
SERVFAIL, or REFUSED when asked questions that they cannot answer or 
don't want to answer.


Quad9 is one such. It is UNFIT for any use by any mail system. It tells 
you lies about DNS, supposedly for what its operators deem to be your 
own good.




--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

Spamassassin reporting IP address is whitelisted by DNSWL.org but DNSWL.org reports it is not

[Steve Dondley]

I'm very, very sorry to beat a dead horse, but I'm deeply confused by 
the "RCVD_IN_DNSWL_HI" rule which appears to be reporting incorrectly on 
my system.


I ran this command:

sudo -u s -- spamassassin -t -d < some_email

It gives me this report:

 pts rule name  description
 -- 
--

 1.2 URIBL_ABUSE_SURBL  Contains an URL listed in the ABUSE SURBL
blocklist
[URIs: bizgrouplinknews.com]
 1.7 URIBL_BLACKContains an URL listed in the URIBL 
blacklist

[URIs: bizgrouplinknews.com]
 2.5 URIBL_DBL_SPAM Contains a spam URL listed in the Spamhaus 
DBL

blocklist
[URIs: bizgrouplinknews.com]
 0.0 RCVD_IN_MSPIKE_L5  RBL: Very bad reputation (-5)
[50.30.46.135 listed in bl.mailspike.net]
-2.0 RCVD_IN_DNSWL_HI   RBL: Sender listed at 
https://www.dnswl.org/,

high trust
[50.30.46.135 listed in list.dnswl.org]
 0.5 BAYES_999  BODY: Bayes spam probability is 99.9 to 100%
[score: 1.]
 3.5 BAYES_99   BODY: Bayes spam probability is 99 to 100%
[score: 1.]
 1.3 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in
bl.spamcop.net
   [Blocked - see 
]

-0.0 SPF_PASS   SPF: sender matches SPF record
 0.0 SPF_HELO_NONE  SPF: HELO does not publish an SPF Record
 2.6 DEAR_FRIENDBODY: Dear Friend? That's not very dear!
 0.0 HTML_MESSAGE   BODY: HTML included in message
 0.1 HTTPS_HTTP_MISMATCHBODY: No description available.
-0.1 DKIM_VALID_AU  Message has a valid DKIM or DK signature 
from

author's domain
 0.1 DKIM_SIGNEDMessage has a DKIM or DK signature, not 
necessarily

valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK 
signature

 0.0 RCVD_IN_MSPIKE_BL  Mailspike blacklisted
 3.5 URI_PHP_REDIR  PHP redirect to different URL (link 
obfuscation)



So it's showing the IP address 50.30.46.135 is whitelisted as shown by 
the RCVD_IN_DNSWL_HI rule.


However, the dnswl.org domain shows that the 50.30.46.135 is *not* 
whitelisted: https://www.dnswl.org/s/?s=50.30.46.135


So what would account for my system reporting it as whitelisted when the 
dnswl.org domain does not report it as whitelisted?

Re: DNSWL overriding bayes_99 and bayes_999 rules

[Bill Cole]

On 10 Apr 2021, at 12:55, Steve Dondley wrote:


You should fix URIBL_BLOCKED first.
You need a local, caching, non-forwarding DNS server for 
SpamAssassin.


Yeah, setting up a DNS server for SA is on my todo list. Thanks.

When you say local, it doesn't have to be on the same machine as 
spamassassin, does it? I assume I can have the DNS server on a local 
network and shared between many machines.


Sure, but it is very simple to stand up a caching, non-forwarding, 
local-only resolver with Unbound or the Knot Resolver (which are 
designed for just that) or even BIND (which has broader applications) on 
most Linux distros, on FreeBSD, or on pre-Catalina MacOS X. It becomes a 
slightly more complicated task when you try to make it serve many 
machines. It also may come to pass that you want to do things at the DNS 
level which are only appropriate for your mail server and not your other 
machines.


--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

Re: DNSWL overriding bayes_99 and bayes_999 rules

[Benny Pedersen]

On 2021-04-10 17:51, Steve Dondley wrote:

I have been looking at this issue a little more. I just grepped my
spam folder. Out of 1000 emails I have flagged as spam, 321 have been
flagged with RCVD_DNSWL_HI, a rule which adds -5 points to the eamil.
That's almost 1 out of 3 emails which seems pretty insane.


Here are the headers from some egregious spam. It scored a whopping
20.8 point despite being flagged with "RCVD_IN_DNSWL_HI."



SPF_SOFTFAIL,SPOOFED_FREEMAIL,SPOOFED_FREEMAIL_NO_RDNS,
SPOOFED_FREEM_REPTO,TVD_SPACE_ENCODED,URIBL_ABUSE_SURBL,URIBL_BLOCKED



while URIBL_BLOCKED :=)

Re: DNSWL overriding bayes_99 and bayes_999 rules

[Benny Pedersen]

On 2021-04-10 17:36, Steve Dondley wrote:


Is anyone else seeing spam getting flagged with RCVD_DNSWL_HI
resulting in so many false positives?


report this ip to dnswl with content as provding evedence, you know 
admins from dnswl.org here recently asked for this ?

Re: DNSWL overriding bayes_99 and bayes_999 rules

[Steve Dondley]

You should fix URIBL_BLOCKED first.
You need a local, caching, non-forwarding DNS server for SpamAssassin.


Yeah, setting up a DNS server for SA is on my todo list. Thanks.

When you say local, it doesn't have to be on the same machine as 
spamassassin, does it? I assume I can have the DNS server on a local 
network and shared between many machines.

Re: DNSWL overriding bayes_99 and bayes_999 rules

[Steve Dondley]

It would be helpful to post an entire actual set of headers --
unmodified -- along with the spamassassin -t report.  I can't figure
out (from what you posted) the IP address of the server that was in
DNSWL_HI that delivered mail to your internal/trusted network.


OK, here is the entire output of this command:

sudo -u s -- spamassassin -t -d < the_spam_email

Note: I've changed the score of RCVD_IN_DNSWL_HI hits to -2.0 from -5.0 
until I get my misconfiguration figured out. Thanks for your patience.





Received: from localhost by email.dondley.com
with SpamAssassin (version 3.4.2);
Sat, 10 Apr 2021 12:41:17 -0400
From: 
=?shift_jis?B?kmqCzI/bkqWKZ5HljHaJ5iBBaXAxMA==?=

To: 
Subject: *SPAM* 
=?shift_jis?B?g0mDk4NpgqqLgYLfgumXQojqlrOT8YLMgZqDZoNKg2CDk4GagvCBSTA5?=

Date: Sat, 10 Apr 2021 18:50:01 +0900
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on 
email.dondley.com

X-Spam-Flag: YES
X-Spam-Level: ***
X-Spam-Status: Yes, score=23.2 required=5.0 tests=BASE64_LENGTH_79_INF,
BAYES_99,BAYES_999,DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,FREEMAIL_REPLYTO,
FREEMAIL_REPLYTO_END_DIGIT,FROM_MISSP_FREEMAIL,FROM_MISSP_REPLYTO,
LOCAL_SPAM_TLD,LOCAL_UNCOMMON_TLD,MISSING_MID,NML_ADSP_CUSTOM_MED,
RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_DNSWL_HI,RCVD_IN_MSPIKE_H2,RCVD_IN_PSBL,
RCVD_IN_RP_RNBL,RCVD_IN_VALIDITY_RPBL,RDNS_NONE,SPF_HELO_SOFTFAIL,
SPF_SOFTFAIL,SPOOFED_FREEMAIL,SPOOFED_FREEMAIL_NO_RDNS,
SPOOFED_FREEM_REPTO,TVD_SPACE_ENCODED shortcircuit=no autolearn=no
autolearn_force=no version=3.4.2
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="--=_6071D52D.C7B255FE"

This is a multi-part message in MIME format.

=_6071D52D.C7B255FE
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit

Spam detection software, running on the system "email.dondley.com",
has identified this incoming email as possible spam.  The original
message has been attached to this so you can view it or label
similar future email.  If you have any questions, see
the administrator of that system for details.

Content preview:  
@„ª{„ª{„ª{„ª{„ª{„ª{„ª{„ª{„ª{„ª{„ª{„ª{„ª{„ª{„ª
   @@@@@@@™‹ÆŠEÅ‚‚̃{ƒ‹ƒ‚ƒ“¬’·Œø‰Ê™ 
@@@@@@@@@@šƒyƒjƒX‘‘åƒTƒvƒŠš



Content analysis details:   (23.2 points, 5.0 required)

 pts rule name  description
 -- 
--
-2.0 RCVD_IN_DNSWL_HI   RBL: Sender listed at 
https://www.dnswl.org/,

high trust
[203.160.71.180 listed in list.dnswl.org]
-0.0 RCVD_IN_MSPIKE_H2  RBL: Average reputation (+2)
[203.160.71.180 listed in wl.mailspike.net]
 2.7 RCVD_IN_PSBL   RBL: Received via a relay in PSBL
[203.160.71.180 listed in psbl.surriel.com]
 3.5 BAYES_99   BODY: Bayes spam probability is 99 to 100%
[score: 1.]
 0.5 BAYES_999  BODY: Bayes spam probability is 99.9 to 100%
[score: 1.]
 2.0 LOCAL_SPAM_TLD Domain originates a lot of spam
 1.0 LOCAL_UNCOMMON_TLD From address is not a common TLD
 1.3 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in
bl.spamcop.net
 [Blocked - see 
]

 1.3 RCVD_IN_VALIDITY_RPBL  RBL: Relay in Validity RPBL,
https://senderscore.org/blocklistlookup/
   [203.160.71.180 listed in 
bl.score.senderscore.com]

 0.0 FREEMAIL_FROM  Sender email is commonly abused enduser mail
provider (qy5cbma-yua06[at]yahoo.co.jp)
 0.2 FREEMAIL_REPLYTO_END_DIGIT Reply-To freemail username ends in
digit (qy5cbma-yua06[at]yahoo.co.jp)
 0.7 SPF_SOFTFAIL   SPF: sender does not match SPF record 
(softfail)

 0.0 DKIM_ADSP_CUSTOM_MED   No valid author signature, adsp_override is
 CUSTOM_MED
 0.7 SPF_HELO_SOFTFAIL  SPF: HELO does not match SPF record 
(softfail)

 1.5 BASE64_LENGTH_79_INF   BODY: base64 encoded email part uses line
length greater than 79 characters
 0.5 MISSING_MIDMissing Message-Id: header
 0.0 RCVD_IN_RP_RNBLRCVD_IN_RP_RNBL renamed to
RCVD_IN_VALIDITY_RPBL, please update local
 rules
 0.8 RDNS_NONE  Delivered to internal network by a host with 
no rDNS

 1.0 FREEMAIL_REPLYTO   Reply-To/From or Reply-To/body contain
different freemails
 0.9 NML_ADSP_CUSTOM_MEDADSP custom_med hit, and not from a mailing
list
 0.0 FROM_MISSP_REPLYTO From misspaced, has Reply-To
 2.5 TVD_SPACE_ENCODED 

Re: DNSWL overriding bayes_99 and bayes_999 rules

[Bill Cole]

On 10 Apr 2021, at 12:19, Steve Dondley wrote:


On 2021-04-10 12:10 PM, Greg Troxel wrote:

Steve Dondley  writes:


Here are the headers from some egregious spam. It scored a whopping
20.8 point despite being flagged with "RCVD_IN_DNSWL_HI."

Return-Path: 
Delivered-To: s...@example.com
Received: from email.example.com
by email.example.com with LMTP
id AnV2NSCZbmCTcQAAB604Gw
(envelope-from )
for ; Thu, 08 Apr 2021 01:48:16 -0400


really?  Those are the headers?


Yes. Why do you ask? Is it unusual that this egregious example of spam 
is on DNSWL_HI?


It's not that, it's the fact that you appear to have provided the 
headers of the 'wrapper' message that SA creates and re-injects when 
report_safe=1  or report_safe=2 rather than the actual headers of the 
original message.




--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

Re: DNSWL overriding bayes_99 and bayes_999 rules

[Greg Troxel]

Steve Dondley  writes:

> On 2021-04-10 12:10 PM, Greg Troxel wrote:
>> Steve Dondley  writes:
>>
>>> Here are the headers from some egregious spam. It scored a whopping
>>> 20.8 point despite being flagged with "RCVD_IN_DNSWL_HI."
>>>
>>> Return-Path: 
>>> Delivered-To: s...@example.com
>>> Received: from email.example.com
>>> by email.example.com with LMTP
>>> id AnV2NSCZbmCTcQAAB604Gw
>>> (envelope-from )
>>> for ; Thu, 08 Apr 2021 01:48:16 -0400
>>
>> really?  Those are the headers?
>
> Yes. Why do you ask? Is it unusual that this egregious example of spam
> is on DNSWL_HI?

I don't see the header added by your MTA with the IP address of the
source.  And example.com is bogus; it looks like you edited things and
removed information.

>> So my advice again is:
>>
>>   Run spamassassin -t on the message so you see the metadata about the
>>   rules like which IP hit and the per-rule score.
>
> I've already done that on selective email messages.

It would be helpful to post an entire actual set of headers --
unmodified -- along with the spamassassin -t report.  I can't figure
out (from what you posted) the IP address of the server that was in
DNSWL_HI that delivered mail to your internal/trusted network.

>>   If you got spam from a sender in DNSWL_HI, report it to dnswl.org.
>>   Give them a week and see if they take the IP out, or what happens,
>> and
>>   tell us how it went.
>
> I plan on it but first:
>
> 1) I want to verify with this list I don't have something
> misconfigured before I report 300+ emails. From what I've read in the
> emails last week, this would be highly unusual.

It would, and if you haven't munged the headers that you posted, things
seem very very odd on your end to me.


signature.asc
Description: PGP signature

Re: learning news from Spamassassin ?

[Bill Cole]

On 10 Apr 2021, at 10:17, RW wrote:


On Sat, 10 Apr 2021 13:23:01 +0200
Matus UHLAR - fantomas wrote:


On 10.04.21 08:58, mau...@gmx.ch wrote:

my spamassassin book are coming from 2004, and possible this arnt
relay up2date.


should be 90% fine.



I didn't know there was a book but I looked it up

"Configure SpamAssassin to work with newer spam-filtering methods such
as Hashcash"


For people who don't get the point here:

DO NOT attempt to configure SpamAssassin to use Hashcash. It was never 
really a good idea, it has never been enabled by default, the SA plugin 
hasn't had any specific attention since 2005, and it turns out that the 
proof of work that no one managed to provide was configuring it. It will 
be going away altogether in v4.0.



2004 is very early, it probably doesn't even cover internal networks
which only went into trunk in November of that year. I'd ignore that
book, there's plenty of online help.


Agreed.

--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

Re: DNSWL overriding bayes_99 and bayes_999 rules

[Arne Jensen]

You do obviously have a very misconfigured system on your end.

Den 10-04-2021 kl. 17:51 skrev Steve Dondley:
>
> X-Spam-Status: Yes, score=20.8 required=5.0 tests=BASE64_LENGTH_79_INF,
>     [...]
>     ***RCVD_IN_DNSWL_HI***,RCVD_IN_PSBL,RCVD_IN_RP_RNBL,RCVD_IN_SBL_CSS,
> RCVD_IN_VALIDITY_RPBL,RCVD_IN_XBL,***RDNS_NONE***,SPF_HELO_SOFTFAIL,
>     [...]

DNSWL does not list any IP addresses without Reverse DNS (PTR), that
also matches with the forward DNS ( see the markings above ! ).


- So maybe you should look at your set up, instead of continuing your
game of claiming false positives with DNSWL?

- Again, maybe you should put up the FULL MESSAGE, instead of only
partial / munged headers?


Maybe then, and only then, there would be much more suggestions for how
to proceed.

-- 
Med venlig hilsen / Kind regards,
Arne Jensen

Re: DNSWL overriding bayes_99 and bayes_999 rules

[Matus UHLAR - fantomas]

Steve Dondley  writes:

Here are the headers from some egregious spam. It scored a whopping
20.8 point despite being flagged with "RCVD_IN_DNSWL_HI."

Return-Path: 
Delivered-To: s...@example.com
Received: from email.example.com
by email.example.com with LMTP
id AnV2NSCZbmCTcQAAB604Gw
(envelope-from )
for ; Thu, 08 Apr 2021 01:48:16 -0400



On 2021-04-10 12:10 PM, Greg Troxel wrote:

really?  Those are the headers?


On 10.04.21 12:19, Steve Dondley wrote:
Yes. Why do you ask? Is it unusual that this egregious example of spam 
is on DNSWL_HI?


there are too few e-mail headers there, and it looks like the mail was
submitted from your machine.
We even don't see the IP that's supposed to be in dnswl.


So my advice again is:

 Run spamassassin -t on the message so you see the metadata about the
 rules like which IP hit and the per-rule score.


I've already done that on selective email messages.


 If you got spam from a sender in DNSWL_HI, report it to dnswl.org.
 Give them a week and see if they take the IP out, or what happens, 
and

 tell us how it went.


I plan on it but first:

1) I want to verify with this list I don't have something 
misconfigured before I report 300+ emails. From what I've read in the 
emails last week, this would be highly unusual.


2) If I do have that many false positives, I need to figure out how to 
bulk report that many of them.


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
If Barbie is so popular, why do you have to buy her friends?

Re: DNSWL overriding bayes_99 and bayes_999 rules

[jwmincy]

Steve Dondley writes:
 > From: Steve Dondley 
 > Date: Sat, 10 Apr 2021 11:51:16 -0400
 > 
 > 
 > > I have been looking at this issue a little more. I just grepped my
 > > spam folder. Out of 1000 emails I have flagged as spam, 321 have been
 > > flagged with RCVD_DNSWL_HI, a rule which adds -5 points to the eamil.
 > > That's almost 1 out of 3 emails which seems pretty insane.
 > 
 > Here are the headers from some egregious spam. It scored a whopping 20.8 
 > point despite being flagged with "RCVD_IN_DNSWL_HI."
 > 
 > Return-Path: 
 > Delivered-To: s...@example.com
 > Received: from email.example.com
 >  by email.example.com with LMTP
 >  id AnV2NSCZbmCTcQAAB604Gw
 >  (envelope-from )
 >  for ; Thu, 08 Apr 2021 01:48:16 -0400
 > Received: by email.example.com (Postfix, from userid 115)
 >  id CDD3D210E1; Thu,  8 Apr 2021 01:48:16 -0400 (EDT)
 > Received: from localhost by email.example.com
 >  with SpamAssassin (version 3.4.2);
 >  Thu, 08 Apr 2021 01:48:16 -0400
 >  From: 
 > DVD   Aip08 
 > To: 
 > Subject: *SPAM* 
 >  AV07
 > Date: Thu, 08 Apr 2021 14:48:09 +0900
 > X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on 
 > email.example.com
 > X-Spam-Flag: YES
 > X-Spam-Level: 
 > X-Spam-Status: Yes, score=20.8 required=5.0 tests=BASE64_LENGTH_79_INF,
 >  DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,FREEMAIL_REPLYTO,
 >  FREEMAIL_REPLYTO_END_DIGIT,FROM_MISSP_FREEMAIL,FROM_MISSP_REPLYTO,
 >  MISSING_MID,NML_ADSP_CUSTOM_MED,RCVD_IN_BL_SPAMCOP_NET,
 >  RCVD_IN_DNSWL_HI,RCVD_IN_PSBL,RCVD_IN_RP_RNBL,RCVD_IN_SBL_CSS,
 >  RCVD_IN_VALIDITY_RPBL,RCVD_IN_XBL,RDNS_NONE,SPF_HELO_SOFTFAIL,
 >  SPF_SOFTFAIL,SPOOFED_FREEMAIL,SPOOFED_FREEMAIL_NO_RDNS,
 >  SPOOFED_FREEM_REPTO,TVD_SPACE_ENCODED,URIBL_ABUSE_SURBL,URIBL_BLOCKED
 >  shortcircuit=no autolearn=unavailable autolearn_force=no version=3.4.2
 > MIME-Version: 1.0
 > Content-Type: multipart/mixed; boundary="--=_606E9920.15B94EAE"
 > Message-Id: <20210408054816.cdd3d21...@email.example.com>
 > 


You should fix URIBL_BLOCKED first.
You need a local, caching, non-forwarding DNS server for SpamAssassin.


I haven't noticed much of any spam hitting DNSWL HI.  I suspect you
have some other configuration issue.

-jeff

Re: DNSWL overriding bayes_99 and bayes_999 rules

[Steve Dondley]

On 2021-04-10 12:10 PM, Greg Troxel wrote:

Steve Dondley  writes:


Here are the headers from some egregious spam. It scored a whopping
20.8 point despite being flagged with "RCVD_IN_DNSWL_HI."

Return-Path: 
Delivered-To: s...@example.com
Received: from email.example.com
by email.example.com with LMTP
id AnV2NSCZbmCTcQAAB604Gw
(envelope-from )
for ; Thu, 08 Apr 2021 01:48:16 -0400


really?  Those are the headers?


Yes. Why do you ask? Is it unusual that this egregious example of spam 
is on DNSWL_HI?




So my advice again is:

  Run spamassassin -t on the message so you see the metadata about the
  rules like which IP hit and the per-rule score.


I've already done that on selective email messages.


  If you got spam from a sender in DNSWL_HI, report it to dnswl.org.
  Give them a week and see if they take the IP out, or what happens, 
and

  tell us how it went.


I plan on it but first:

1) I want to verify with this list I don't have something misconfigured 
before I report 300+ emails. From what I've read in the emails last 
week, this would be highly unusual.


2) If I do have that many false positives, I need to figure out how to 
bulk report that many of them.

Re: DNSWL overriding bayes_99 and bayes_999 rules

[Greg Troxel]

Steve Dondley  writes:

> Here are the headers from some egregious spam. It scored a whopping
> 20.8 point despite being flagged with "RCVD_IN_DNSWL_HI."
>
> Return-Path: 
> Delivered-To: s...@example.com
> Received: from email.example.com
>   by email.example.com with LMTP
>   id AnV2NSCZbmCTcQAAB604Gw
>   (envelope-from )
>   for ; Thu, 08 Apr 2021 01:48:16 -0400

really?  Those are the headers?

So my advice again is:

  Run spamassassin -t on the message so you see the metadata about the
  rules like which IP hit and the per-rule score.

  If you got spam from a sender in DNSWL_HI, report it to dnswl.org.
  Give them a week and see if they take the IP out, or what happens, and
  tell us how it went.



signature.asc
Description: PGP signature

Re: DNSWL overriding bayes_99 and bayes_999 rules

[Steve Dondley]

I have been looking at this issue a little more. I just grepped my
spam folder. Out of 1000 emails I have flagged as spam, 321 have been
flagged with RCVD_DNSWL_HI, a rule which adds -5 points to the eamil.
That's almost 1 out of 3 emails which seems pretty insane.


Here are the headers from some egregious spam. It scored a whopping 20.8 
point despite being flagged with "RCVD_IN_DNSWL_HI."


Return-Path: 
Delivered-To: s...@example.com
Received: from email.example.com
by email.example.com with LMTP
id AnV2NSCZbmCTcQAAB604Gw
(envelope-from )
for ; Thu, 08 Apr 2021 01:48:16 -0400
Received: by email.example.com (Postfix, from userid 115)
id CDD3D210E1; Thu,  8 Apr 2021 01:48:16 -0400 (EDT)
Received: from localhost by email.example.com
with SpamAssassin (version 3.4.2);
Thu, 08 Apr 2021 01:48:16 -0400
From: 
=?shift_jis?B?i9aSZoLMl6BEVkSP7pXxIEFpcDA4jYY=?=

To: 
Subject: *SPAM* 
=?shift_jis?B?lrOPQ5Czi8mU6Zesj2+DVoOKgVuDWYFFg4KDVYNDg06UaonzgUWXTJa8QVaPl5dEgXmXoIOCg22JroF6MDc=?=

Date: Thu, 08 Apr 2021 14:48:09 +0900
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on 
email.example.com

X-Spam-Flag: YES
X-Spam-Level: 
X-Spam-Status: Yes, score=20.8 required=5.0 tests=BASE64_LENGTH_79_INF,
DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,FREEMAIL_REPLYTO,
FREEMAIL_REPLYTO_END_DIGIT,FROM_MISSP_FREEMAIL,FROM_MISSP_REPLYTO,
MISSING_MID,NML_ADSP_CUSTOM_MED,RCVD_IN_BL_SPAMCOP_NET,
RCVD_IN_DNSWL_HI,RCVD_IN_PSBL,RCVD_IN_RP_RNBL,RCVD_IN_SBL_CSS,
RCVD_IN_VALIDITY_RPBL,RCVD_IN_XBL,RDNS_NONE,SPF_HELO_SOFTFAIL,
SPF_SOFTFAIL,SPOOFED_FREEMAIL,SPOOFED_FREEMAIL_NO_RDNS,
SPOOFED_FREEM_REPTO,TVD_SPACE_ENCODED,URIBL_ABUSE_SURBL,URIBL_BLOCKED
shortcircuit=no autolearn=unavailable autolearn_force=no version=3.4.2
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="--=_606E9920.15B94EAE"
Message-Id: <20210408054816.cdd3d21...@email.example.com>

Re: DNSWL overriding bayes_99 and bayes_999 rules

[Steve Dondley]

On 2021-04-06 11:48 AM, Steve Dondley wrote:

I have emails that have been flagged as spam in the past but that are
still getting through, presumably because the servers are on some
DNSWL.

Example:

X-Spam-Status: No, score=0.9 required=5.0 tests=BAYES_99,BAYES_999,
DATE_IN_PAST_03_06,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,

HTML_IMAGE_RATIO_02,HTML_MESSAGE,RCVD_IN_DNSWL_HI,RCVD_IN_MSPIKE_H2,

SPF_HELO_NONE,SPF_SOFTFAIL shortcircuit=no autolearn=no
autolearn_force=no version=3.4.2

What's the recommended way to handle these? Do I turn on shortcircuit?
Do I bump up the score for BAYES_99, BAYES_999? Or might there be a
way to ignore DNSWL scores if they have a high bayes score?


I have been looking at this issue a little more. I just grepped my spam 
folder. Out of 1000 emails I have flagged as spam, 321 have been flagged 
with RCVD_DNSWL_HI, a rule which adds -5 points to the eamil. That's 
almost 1 out of 3 emails which seems pretty insane.


Is anyone else seeing spam getting flagged with RCVD_DNSWL_HI resulting 
in so many false positives?

Re: learning news from Spamassassin ?

[RW]

On Sat, 10 Apr 2021 13:23:01 +0200
Matus UHLAR - fantomas wrote:

> On 10.04.21 08:58, mau...@gmx.ch wrote:
> >my spamassassin book are coming from 2004, and possible this arnt
> >relay up2date.  
> 
> should be 90% fine.


I didn't know there was a book but I looked it up

"Configure SpamAssassin to work with newer spam-filtering methods such
as Hashcash"


2004 is very early, it probably doesn't even cover internal networks
which only went into trunk in November of that year. I'd ignore that
book, there's plenty of online help.

Re: OT: is sorbs.net sleeping ?

[Benny Pedersen]

On 2021-04-10 15:59, RW wrote:

On Sat, 10 Apr 2021 15:44:54 +0200
Benny Pedersen wrote:



dont use public dns servers ever, free or not



It's not about using public caches. They are going to block look-ups
from generic rDNS as well. I think they are already blocking some VPS
address blocks.


and if users of dqs do try that dqs key is shared

the first dqs rule set had that problem in _REPORT_

hope rules in 4.x.x will handle this in generic without using meta rules

Re: OT: is sorbs.net sleeping ?

[RW]

On Sat, 10 Apr 2021 15:44:54 +0200
Benny Pedersen wrote:


> dont use public dns servers ever, free or not
> 

It's not about using public caches. They are going to block look-ups
from generic rDNS as well. I think they are already blocking some VPS
address blocks.

Re: OT: is sorbs.net sleeping ?

[Benny Pedersen]

On 2021-04-10 15:28, RW wrote:

On Sat, 10 Apr 2021 08:56:19 -0400
Rob McEwen wrote:


On 4/10/2021 6:55 AM, Jared Hall wrote:
> Rob, I gotta say that I am impressed with the whole Spamhaus-dqs
> program and their use of customer keyed DNS zone queries.  Seems to
> be the way around the client DNS forwarder issues.  How are you
> guys at Invaluement tracking in that area?

I'm not sure I'm understanding what you're saying? Are you referring
to the fact that their paid customers doing direct queries (NOT the
free stuff!) - use zone names that have a unique key embedded into
the actual zone - so that the queries can then be distinguished by
this unique key?


It's not just paid customers, anyone can register.


and use there own key with public dns servers, hillerious

spamassassin shows the dqs key with default rules, so workaround is meta 
rule


dont use public dns servers ever, free or not

after all its not free

can i get a ansver on sorbs ?, is it time to not use sorbs in 
spamassassin or is there a way to contakt sorbs ?, i have giving up 
trying :(


hopefully dnsbl owners is professionel people until it shown thay are 
not

Re: OT: is sorbs.net sleeping ?

[RW]

On Sat, 10 Apr 2021 08:56:19 -0400
Rob McEwen wrote:

> On 4/10/2021 6:55 AM, Jared Hall wrote:
> > Rob, I gotta say that I am impressed with the whole Spamhaus-dqs 
> > program and their use of customer keyed DNS zone queries.  Seems to
> > be the way around the client DNS forwarder issues.  How are you
> > guys at Invaluement tracking in that area?  
> 
> I'm not sure I'm understanding what you're saying? Are you referring
> to the fact that their paid customers doing direct queries (NOT the
> free stuff!) - use zone names that have a unique key embedded into
> the actual zone - so that the queries can then be distinguished by
> this unique key? 

It's not just paid customers, anyone can register.
 

Re: OT: is sorbs.net sleeping ?

[Rob McEwen]

On 4/10/2021 6:55 AM, Jared Hall wrote:
Rob, I gotta say that I am impressed with the whole Spamhaus-dqs 
program and their use of customer keyed DNS zone queries.  Seems to be 
the way around the client DNS forwarder issues.  How are you guys at 
Invaluement tracking in that area?


I'm not sure I'm understanding what you're saying? Are you referring to 
the fact that their paid customers doing direct queries (NOT the free 
stuff!) - use zone names that have a unique key embedded into the actual 
zone - so that the queries can then be distinguished by this unique key? 
- thus eliminating the need to use the client's local DNS servers' 
public IP as the method of allowing/denying direct queries? Is that what 
you're referring to?



Seems to be the way around the client DNS forwarder issues


If I'm correct about what you meant - then yes - this eliminates 
problems that used to happen when trying to track customers, and 
permission, by IP - because when tracking by an embedded code - then it 
doesn't matter from WHERE the queries come - and queries that come from 
public DNS servers (8.8.8.8 or 1.1.1.1) - can be distinguished one from 
the other - whereas when not doing this - it's impossible to tell 
distinguish the queries from each other and know who is doing them. This 
became especially important because so often the default caching DNS 
server gets auto-flipped to 8.8.8.8, sometimes without the IT person's 
knowledge! And many IT people think that pointing to 8.8.8.8 is the 
textbook way to setup DNS - and have never even heard of things like BIND.


Is THAT what you're talking about?

If so, at invaluement, we've been doing this for 3 years now - but we 
still have a lot of work to do in migrating many long-time customers 
over to our new system. And it was developed before I even knew that 
Spamhaus was doing it this way, and  this involved some extremely 
complex custom modifications of rbldnsd (I couldn't afford to hire an 
expensive high-quality C++ programmer at the time - so it took me about 
100 hours of very intense programming to do that! It didn't help that 
I'm not very good at C++!). I'm not even sure when Spamhaus started this.


Our new system for doing this now involves 86 servers in 43 cities 
around the world - which enables our clients to get their queries 
answered much faster due to accessing an invaluement DNS server with an 
extremely close geolocation. Queries then tend to get answered in a very 
low number of milliseconds - often <10ms.


-- Rob McEwen https://www.invaluement.com +1 (478) 475-9032

Re: learning news from Spamassassin ?

[Matus UHLAR - fantomas]

On 10.04.21 08:58, mau...@gmx.ch wrote:

my spamassassin book are coming from 2004, and possible this arnt relay
up2date.


should be 90% fine.


I need to refresh the update and build the own rules.


apparently new install, preferrably from your OS/distribution.

read /etc/spamassassin/*.pre files and uncomment as needed.
razor, pyzor, DCC, SPF and other plugins may need external
packages/libraries.

most of distros keep rules up-to-date by using sa-update daily (or,
nightly).


I read now meny topics from apache - spamassassin site, and it's time to
refresh

Me point to filter granularity.


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Linux IS user friendly, it's just selective who its friends are...

Re: OT: is sorbs.net sleeping ?

[Jared Hall]

(you might be disappointed with SORBS in those areas too? - that's fine 
- I'm just trying to clarify that overly judging a DNSBL based on 
/*particular*/ false negatives can be overly harsh and might miss the 
good things that a DNSBL has to offer)


Probably not that.  It is just SORBS.  Like when a friend gets you 
kicked out of a bar for trouble you didn't cause:


"I GOT SORBED."

Rob, I gotta say that I am impressed with the whole Spamhaus-dqs program 
and their use of customer keyed DNS zone queries.  Seems to be the way 
around the client DNS forwarder issues.  How are you guys at Invaluement 
tracking in that area?  I saw some esp stuff on Github.


-- Jared Hall