Re: Fwd: [mailop] SORBS Closing.

[J Doe]

On 2024-06-05 04:44, Rob McEwen via users wrote:


 From "Frido Otten" mailto:fr...@0tten.nl>>


So is there anything that needs to be done to prevent false positives
happening right after the shutdown?



They said they were emptying the zone files, not actually "listing the
world" - so this shouldn't cause false any positives - but might cause
some false negatives, especially for anyone who was overly relying on
SORBS in their spam filtering? But yet - after some years - lists like
this - once they've been dead for many many years - do /sometimes/ "list
the world" as a final push to get others to stop using them - if that
even ever happens with SORBS? And if it ever did, I doubt that would
happen anytime soon.

But definately make sure that your spam filter is ONLY ever acting on
specific and valid SORBS return codes - and NOT treating ANY query being
resolved that isn't NXDOMAIN - as a listing. Anyone */misusing/* SORBS
in that way - might one day have a very bad day. But that's a general
truth for all DSNBLs - make sure your rules/setup for its usage only
treat it as a valid listing when specific return codes are returned,
that match that DNSBL's instructions, and thus */NOT/* taking action
just because /some /IP address was resolved by the query.

/(I think any built-in/default SpamAssassin rules for SORBS - already
does all of this correctly.)/

Rob McEwen, invaluement


Hi Rob and list,

Speaking as a small user of SORBS via SpamAssassin 4.0, I assume the
correct response to disable use of SORBS is to place the following in my
local.cf file:

dns_query_restriction deny sorbs.net

Is that correct and is there any additional portions of local.cf I need
to configure so that I am no longer consulting SORBS ?

Thanks,

- J

AW: RCVD_IN_RP_CERTIFIED always -3

[hostmaster]

Thanks a lot Kris.

I just got the latest rules. 
I'm okay with poor performance for some of the rules as there isn't much
load on the related system. 
And yes, you're right, on Ubuntu 20.04.06 the rules are installed in
/usr/share/spamassassin. 
sa-update has placed the updated rules in /var/lib/spamassassin. I kept
usr/share/spamassassin for the moment as if I got it right, /var/lib will
have priority over /usr/share.
Seems to work so far, however I will check the logs in a couple of days to
validate it's actually running smoothly. 

Mark



-Ursprüngliche Nachricht-
Von: Kris Deugau [mailto:kdeu...@vianet.ca] 
Gesendet: Donnerstag, 6. Juni 2024 19:48
An: users@spamassassin.apache.org
Betreff: Re: RCVD_IN_RP_CERTIFIED always -3

hostmas...@audiogen.ch wrote:
> I found the related 
> configuration in 20_dnsbl_tests.cf:
> 
> /# 
>
---/
> 
> /# Return Path Certified:/
> 
> /# https://www.returnpath.net/internetserviceprovider/certification//
> 
> /# (replaces RCVD_IN_BSP_TRUSTED, RCVD_IN_BSP_OTHER, 
> RCVD_IN_SSC_TRUSTED_COI)/
> 
> /header RCVD_IN_RP_CERTIFIED eval:check_rbl_txt('ssc-firsttrusted', 
> 'sa-trusted.bondedsender.org.')/
> 
> /describe RCVD_IN_RP_CERTIFIED   Sender in ReturnPath Certified - 
> Contact cert...@returnpath.net/
> 
> /tflags RCVD_IN_RP_CERTIFIED net nice/
> 
> /reuse RCVD_IN_RP_CERTIFIED/

These were renamed/rebranded some time ago.  Run sa-update to get 
current rules.

Note that the live rules your system references should be under 
/var/lib/spamassassin on most systems.  Some distribution packages will 
also ship a rules snapshot in /usr/share/spamassassin, however these 
should be considered more or less unusable unless you are trying to run 
SA in a very extremely security-restricted environment.

> Regarding the SpamAssassin 4.x rules - are they backward compatible to
3.4.4?

Pretty much, although rules that rely on new features won't work well or 
at all, and some rules will have poor performance as some of the code 
that extracts data they reference has been updated and refined.

-kgd

Re: RCVD_IN_RP_CERTIFIED always -3

[Kris Deugau]

hostmas...@audiogen.ch wrote:
I found the related 
configuration in 20_dnsbl_tests.cf:


/# 
---/


/# Return Path Certified:/

/# https://www.returnpath.net/internetserviceprovider/certification//

/# (replaces RCVD_IN_BSP_TRUSTED, RCVD_IN_BSP_OTHER, 
RCVD_IN_SSC_TRUSTED_COI)/


/header RCVD_IN_RP_CERTIFIED eval:check_rbl_txt('ssc-firsttrusted', 
'sa-trusted.bondedsender.org.')/


/describe RCVD_IN_RP_CERTIFIED   Sender in ReturnPath Certified - 
Contact cert...@returnpath.net/


/tflags RCVD_IN_RP_CERTIFIED net nice/

/reuse RCVD_IN_RP_CERTIFIED/


These were renamed/rebranded some time ago.  Run sa-update to get 
current rules.


Note that the live rules your system references should be under 
/var/lib/spamassassin on most systems.  Some distribution packages will 
also ship a rules snapshot in /usr/share/spamassassin, however these 
should be considered more or less unusable unless you are trying to run 
SA in a very extremely security-restricted environment.



Regarding the SpamAssassin 4.x rules - are they backward compatible to 3.4.4?


Pretty much, although rules that rely on new features won't work well or 
at all, and some rules will have poor performance as some of the code 
that extracts data they reference has been updated and refined.


-kgd

AW: RCVD_IN_RP_CERTIFIED always -3

[hostmaster]

Thanks for your answer Harald. 

Regarding "there is no such configuration option in SpamAssassin":  The conf 
snipplet I posted below comes from the repository, however it's an older 
version, which still is supported by Ubuntu 20.04.06 LTS and can be installed 
from their related archive (at least my rules where last updated in March 23). 
https://github.com/apache/spamassassin/blob/spamassassin_release_3_4_4/trunk-only/rules/20_dnsbl_tests.cf
 (the same is used up to 3.4.6)
I should have written I'm on an older Ubuntu, might have helped to avoid 
confusion. 

Regarding the SpamAssassin 4.x rules - are they backward compatible to 3.4.4?

Thanks a lot
Mark





-Ursprüngliche Nachricht-
Von: Reindl Harald (privat) [mailto:ha...@rhsoft.net] 
Gesendet: Donnerstag, 6. Juni 2024 16:29
An: hostmas...@audiogen.ch; users@spamassassin.apache.org
Betreff: Re: RCVD_IN_RP_CERTIFIED always -3

"RCVD_IN_RP_CERTIFIED" don't exist anywhere and the stuff which exsists 
is correctly using response-codes

the nonsense below would also trigger with 127.255.255.255 responses 
which indicates you are using a shared DNS - complain to the individual 
which was writing blacklist/whitelist rules without response codes - 
that don't happen in any official rules because it's plain wrong

header RCVD_IN_VALIDITY_CERTIFIED eval:check_rbl('ssc-firsttrusted', 
'sa-trusted.bondedsender.org.', '^127\.0\.0\.')
describe RCVD_IN_VALIDITY_CERTIFIED   Sender in Validity Certification - 
Contact certificat...@validity.com
tflags RCVD_IN_VALIDITY_CERTIFIED net nice publish
reuse RCVD_IN_VALIDITY_CERTIFIED  RCVD_IN_RP_CERTIFIED
header RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 
eval:check_rbl('ssc-firsttrusted', 'sa-trusted.bondedsender.org.', 
'127.255.255.255')
describe RCVD_IN_VALIDITY_CERTIFIED_BLOCKED  ADMINISTRATOR NOTICE: The 
query to Validity was blocked.  See 
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more 
information.
tflags RCVD_IN_VALIDITY_CERTIFIED_BLOCKEDnet publish
reuse RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 
RCVD_IN_VALIDITY_CERTIFIED_BLOCKED

Am 06.06.24 um 14:13 schrieb hostmas...@audiogen.ch:
> Every email postfix receives gets a RCVD_IN_RP_CERTIFIED=-3 score. This 
> leads to SPAM passing the filter.
> 
> My findings so far
> 
>  From what I think I understood, RCVD_IN_RP_CERTIFIED checks against a 
> list of “trusted” servers, which are considered to not send SPAM. The 
> list is maintained by “Return Path” (Validity). I found the related 
> configuration in 20_dnsbl_tests.cf:
> 
> /# 
> ---/
> 
> /# Return Path Certified:/
> 
> /# https://www.returnpath.net/internetserviceprovider/certification//
> 
> /# (replaces RCVD_IN_BSP_TRUSTED, RCVD_IN_BSP_OTHER, 
> RCVD_IN_SSC_TRUSTED_COI)/
> 
> /header RCVD_IN_RP_CERTIFIED eval:check_rbl_txt('ssc-firsttrusted', 
> 'sa-trusted.bondedsender.org.')/
> 
> /describe RCVD_IN_RP_CERTIFIED   Sender in ReturnPath Certified - 
> Contact cert...@returnpath.net/
> 
> /tflags RCVD_IN_RP_CERTIFIED net nice/
> 
> /reuse RCVD_IN_RP_CERTIFIED/
> 
> It seems Returnpath/Validity have gone out of service  - is this 
> correct? Neither the URL above is working nor can I resolve 
> sa-trusted.bondedsender.org. Even worse, sa-trusted.bondedsender.org is 
> blacklisted by UCEPROTECTL3.
> 
> If RP is dead, is there an alternative? Or am I missing something here?

RCVD_IN_RP_CERTIFIED always -3

[hostmaster]

Hi all

 

Setup

Postfix with amavis, spamassassin and pyzor

 

Problem

Every email postfix receives gets a RCVD_IN_RP_CERTIFIED=-3 score. This
leads to SPAM passing the filter.

 

My findings so far

>From what I think I understood, RCVD_IN_RP_CERTIFIED checks against a list
of "trusted" servers, which are considered to not send SPAM. The list is
maintained by "Return Path" (Validity). I found the related configuration in
20_dnsbl_tests.cf:

 

#
---

# Return Path Certified:

# https://www.returnpath.net/internetserviceprovider/certification/

# (replaces RCVD_IN_BSP_TRUSTED, RCVD_IN_BSP_OTHER, RCVD_IN_SSC_TRUSTED_COI)

header RCVD_IN_RP_CERTIFIED eval:check_rbl_txt('ssc-firsttrusted',
'sa-trusted.bondedsender.org.')

describe RCVD_IN_RP_CERTIFIED   Sender in ReturnPath Certified - Contact
cert...@returnpath.net

tflags RCVD_IN_RP_CERTIFIED net nice

reuse RCVD_IN_RP_CERTIFIED

 

It seems Returnpath/Validity have gone out of service  - is this correct?
Neither the URL above is working nor can I resolve
sa-trusted.bondedsender.org. Even worse, sa-trusted.bondedsender.org is
blacklisted by UCEPROTECTL3.

 

If RP is dead, is there an alternative? Or am I missing something here?

 

Thanks a lot for your support,

Mark

 

 

 

 

Re: Lots of FN because of VALIDITY* rules

[postgarage Graz IT]

On 6/5/24 13:14, Matus UHLAR - fantomas wrote:

On 2024-06-03 at 08:35:32 UTC-0400 (Mon, 3 Jun 2024 14:35:32 +0200)
postgarage Graz IT 
is rumored to have said:
I think that the active.list file should be updated, when there 
are new rules, shouldn't it?


On 03.06.24 08:52, Bill Cole wrote:
It is updated where it is actually used, on the ASF rule 
maintenance system. It is irrelevant to an operational deployment.


I have no idea why Debian installs that file at all.



On 6/5/24 09:17, Matus UHLAR - fantomas wrote:
It does not, I guess that the OP did because of misunderstanding of 
what it does.



On 6/5/24 11:14, postgarage Graz IT wrote:
No I didn't. Please have a look at 
https://packages.debian.org/bookworm/all/spamassassin/filelist where 
you can clearly see, that it is included in Debian's SA package.


yes, /usr/share/spamassassin/active.list is included, but there's none 
in /var/lib/spamassassin/


Yes, that's what I meant. Sorry for the confusion.



As was already mentioned, it's not used by default.
there was apparently come confusion what's in /var/lib/spamassassin/ on 
Debian



I can only guess that the rules were not fresh enough or OP 
installed obsolete/invalid rules there.


The first thing I did was to check if the updates worked (they did) 
neither did I install any rules myself.


On 05.06.24 12:38, postgarage Graz IT wrote:
OK, after having a second look, I take that claim back. It might be 
that I ran sa-update by manually myself (which works) but maybe does 
not run automatically.


you should run this as user debian-spamd, or let cron handle that. 
Otherwise, you will create files cron will be unable to overwrite, which 
may also cause problems (and may have caused yours)


drwxr-xr-x 5 debian-spamd debian-spamd 4096 Nov 27  2023 
/var/lib/spamassassin/
drwxr-xr-x 3 debian-spamd debian-spamd 4096 Jun  4 02:32 
/var/lib/spamassassin/4.00//
drwxr-xr-x 2 debian-spamd debian-spamd 4096 Jun  4 02:32 
/var/lib/spamassassin/4.00/updates_spamassassin_org/



you can also run "chown -R debian-spamd: /var/lib/spamassassin/" to fix it.


Ah, thanks a lot, I'm an idot. That's quite obvious.

Re: Lots of FN because of VALIDITY* rules

[Matus UHLAR - fantomas]

On 2024-06-03 at 08:35:32 UTC-0400 (Mon, 3 Jun 2024 14:35:32 +0200)
postgarage Graz IT 
is rumored to have said:
I think that the active.list file should be updated, when 
there are new rules, shouldn't it?


On 03.06.24 08:52, Bill Cole wrote:
It is updated where it is actually used, on the ASF rule 
maintenance system. It is irrelevant to an operational 
deployment.


I have no idea why Debian installs that file at all.



On 6/5/24 09:17, Matus UHLAR - fantomas wrote:
It does not, I guess that the OP did because of misunderstanding 
of what it does.



On 6/5/24 11:14, postgarage Graz IT wrote:
No I didn't. Please have a look at 
https://packages.debian.org/bookworm/all/spamassassin/filelist where 
you can clearly see, that it is included in Debian's SA package.


yes, /usr/share/spamassassin/active.list is included, but there's none in 
/var/lib/spamassassin/


As was already mentioned, it's not used by default.
there was apparently come confusion what's in /var/lib/spamassassin/ on 
Debian



I can only guess that the rules were not fresh enough or OP 
installed obsolete/invalid rules there.


The first thing I did was to check if the updates worked (they did) 
neither did I install any rules myself.


On 05.06.24 12:38, postgarage Graz IT wrote:
OK, after having a second look, I take that claim back. It might be 
that I ran sa-update by manually myself (which works) but maybe does 
not run automatically.


you should run this as user debian-spamd, or let cron handle that. 
Otherwise, you will create files cron will be unable to overwrite, which may 
also cause problems (and may have caused yours)


drwxr-xr-x 5 debian-spamd debian-spamd 4096 Nov 27  2023 /var/lib/spamassassin/
drwxr-xr-x 3 debian-spamd debian-spamd 4096 Jun  4 02:32 
/var/lib/spamassassin/4.00//
drwxr-xr-x 2 debian-spamd debian-spamd 4096 Jun  4 02:32 
/var/lib/spamassassin/4.00/updates_spamassassin_org/


you can also run "chown -R debian-spamd: /var/lib/spamassassin/" to fix it.



--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Linux IS user friendly, it's just selective who its friends are...

Re: Lots of FN because of VALIDITY* rules

[postgarage Graz IT]

On 6/5/24 11:14, postgarage Graz IT wrote:



On 6/5/24 09:17, Matus UHLAR - fantomas wrote:

On 2024-06-03 at 08:35:32 UTC-0400 (Mon, 3 Jun 2024 14:35:32 +0200)
postgarage Graz IT 
is rumored to have said:
I think that the active.list file should be updated, when there are 
new rules, shouldn't it?


On 03.06.24 08:52, Bill Cole wrote:
It is updated where it is actually used, on the ASF rule maintenance 
system. It is irrelevant to an operational deployment.


I have no idea why Debian installs that file at all.


It does not, I guess that the OP did because of misunderstanding of 
what it does.


No I didn't. Please have a look at 
https://packages.debian.org/bookworm/all/spamassassin/filelist where you 
can clearly see, that it is included in Debian's SA package.




I can only guess that the rules were not fresh enough or OP installed 
obsolete/invalid rules there.


The first thing I did was to check if the updates worked (they did) 
neither did I install any rules myself.


OK, after having a second look, I take that claim back. It might be that 
I ran sa-update by manually myself (which works) but maybe does not run 
automatically.

Re: Lots of FN because of VALIDITY* rules

[postgarage Graz IT]

On 6/5/24 09:17, Matus UHLAR - fantomas wrote:

On 2024-06-03 at 08:35:32 UTC-0400 (Mon, 3 Jun 2024 14:35:32 +0200)
postgarage Graz IT 
is rumored to have said:
I think that the active.list file should be updated, when there are 
new rules, shouldn't it?


On 03.06.24 08:52, Bill Cole wrote:
It is updated where it is actually used, on the ASF rule maintenance 
system. It is irrelevant to an operational deployment.


I have no idea why Debian installs that file at all.


It does not, I guess that the OP did because of misunderstanding of what 
it does.


No I didn't. Please have a look at 
https://packages.debian.org/bookworm/all/spamassassin/filelist where you 
can clearly see, that it is included in Debian's SA package.




I can only guess that the rules were not fresh enough or OP installed 
obsolete/invalid rules there.


The first thing I did was to check if the updates worked (they did) 
neither did I install any rules myself.

Re: Fwd: [mailop] SORBS Closing.

[Rob McEwen via users]

From "Frido Otten" 
So is there anything that needs to be done to prevent false positives 
happening right after the shutdown?




They said they were emptying the zone files, not actually "listing the 
world" - so this shouldn't cause false any positives - but might cause 
some false negatives, especially for anyone who was overly relying on 
SORBS in their spam filtering? But yet - after some years - lists like 
this - once they've been dead for many many years - do sometimes "list 
the world" as a final push to get others to stop using them - if that 
even ever happens with SORBS? And if it ever did, I doubt that would 
happen anytime soon.


But definately make sure that your spam filter is ONLY ever acting on 
specific and valid SORBS return codes - and NOT treating ANY query being 
resolved that isn't NXDOMAIN - as a listing. Anyone misusing SORBS in 
that way - might one day have a very bad day. But that's a general truth 
for all DSNBLs - make sure your rules/setup for its usage only treat it 
as a valid listing when specific return codes are returned, that match 
that DNSBL's instructions, and thus NOT taking action just because some 
IP address was resolved by the query.


(I think any built-in/default SpamAssassin rules for SORBS - already 
does all of this correctly.)


Rob McEwen, invaluement

Re: [mailop] SORBS Closing.

[Michelle Sullivan]

Nothing will *need* to be done.SORBS *should* be removed from all configurations at the earliest opportunity.SORBS will be shut down properly with the DNS servers and zones returning delagation and empty zones for multiple years (should be 10+.. but that depends on whether Proofpoint exists in 10 years… just like any other company.)Michelle Sullivanhttp://www.mhix.org/If we don't find a way out of this soon, I'm gonna lose it. Lose it... It means go crazy, nuts, insane, bonzo, no longer in possession of ones faculties, three fries short of a Happy Meal, wacko!On 5 Jun 2024, at 18:14, Frido Otten  wrote:

  

  
  
A little heads-up from the MailOp mailinglist. 
So is there anything that needs to be done to prevent false
  positives happening right after the shutdown?


  
   Doorgestuurd bericht 
  

  
Onderwerp:

[mailop] SORBS Closing.
  
  
Datum: 
Wed, 05 Jun 2024 10:36:58 +1000
  
  
Van: 
Michelle Sullivan via mailop 
  
  
Antwoord-naar:

Michelle Sullivan 
  
  
Aan: 
mailop 
  

  
  
  
  For those that haven't heard.  Proofpoint is retiring SORBS
  effective immediately(ish).
  
  Zones will be emptied shortly and within a few weeks the SORBS
  domain will be parked on dedicated "decommissioning" servers.
  
  I am being made redundant as part of the shutdown and my last day
  will be 30th June 2024.  I will be looking for new positions
  following that.
  
  I would like to thank all the SORBS supporters over the years and
  Proofpoint for keeping it going for the community for the last 13
  years.
  
  Best regards,
  
  Michelle Sullivan
  SORBS.
  
  -- 
Michelle Sullivan
http://www.mhix.org/

___
mailop mailing list
mai...@mailop.org
https://list.mailop.org/listinfo/mailop


  

Fwd: [mailop] SORBS Closing.

[Frido Otten]

A little heads-up from the MailOp mailinglist.

So is there anything that needs to be done to prevent false positives 
happening right after the shutdown?




 Doorgestuurd bericht 
Onderwerp:  [mailop] SORBS Closing.
Datum:  Wed, 05 Jun 2024 10:36:58 +1000
Van:Michelle Sullivan via mailop 
Antwoord-naar:  Michelle Sullivan 
Aan:mailop 



For those that haven't heard.  Proofpoint is retiring SORBS effective 
immediately(ish).


Zones will be emptied shortly and within a few weeks the SORBS domain 
will be parked on dedicated "decommissioning" servers.


I am being made redundant as part of the shutdown and my last day will 
be 30th June 2024.  I will be looking for new positions following that.


I would like to thank all the SORBS supporters over the years and 
Proofpoint for keeping it going for the community for the last 13 years.


Best regards,

Michelle Sullivan
SORBS.

--
Michelle Sullivan
http://www.mhix.org/

___
mailop mailing list
mai...@mailop.org
https://list.mailop.org/listinfo/mailop



OpenPGP_0xCCDCFB22C59E9DD2.asc
Description: OpenPGP public key


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: Lots of FN because of VALIDITY* rules

[Matus UHLAR - fantomas]

On 2024-06-03 at 08:35:32 UTC-0400 (Mon, 3 Jun 2024 14:35:32 +0200)
postgarage Graz IT 
is rumored to have said:
I think that the active.list file should be updated, when there are 
new rules, shouldn't it?


On 03.06.24 08:52, Bill Cole wrote:
It is updated where it is actually used, on the ASF rule maintenance 
system. It is irrelevant to an operational deployment.


I have no idea why Debian installs that file at all.


It does not, I guess that the OP did because of misunderstanding of what it 
does.


I can only guess that the rules were not fresh enough or OP installed 
obsolete/invalid rules there.


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
The only substitute for good manners is fast reflexes.

Re: DKIM length 'l=' tag

[Matus UHLAR - fantomas]

On 03.06.24 11:16, Marc wrote:
Hi Andrew, this is a bit of topic, I posted this a while ago on the mailing 
list.  But did you notice by any chance that eg.  hotmail.com is failing 
every dkim verification (except their sender rewritten messages)?


I have checked yesterdays logs on one machine:

Jun  4 08:57:50 proxy1 opendmarc[1815]: 4VthHc4zf1zMlJH: hotmail.com pass
Jun  4 09:22:58 proxy1 opendmarc[1815]: 4Vthrc0mrNzMlFy: hotmail.com pass
Jun  4 12:25:10 proxy1 opendmarc[1815]: 4Vtmts4GXGzMlM0: outlook.com fail
Jun  4 12:32:17 proxy1 opendmarc[1815]: 4Vtn336J76zMl7T: hotmail.com pass
Jun  4 12:36:04 proxy1 opendmarc[1815]: 4Vtn7R1B6pzMlCK: hotmail.com pass
Jun  4 12:39:01 proxy1 opendmarc[1815]: 4VtnBr5mfRzMlB6: hotmail.com pass
Jun  4 17:36:30 proxy1 opendmarc[1815]: 4Vtvp4063FzMlM4: hotmail.com pass
Jun  4 21:24:20 proxy1 opendmarc[1815]: 4Vv0rz0TXJzMlLw: outlook.com pass
Jun  4 21:30:55 proxy1 opendmarc[1815]: 4Vv10b0BFZzMlLv: outlook.com pass

The failing 4Vtmts4GXGzMlM0  is DSN, which microsoft software (including 
on-premise exchange servers) seems not to dkim-sign.


I guess that the From: header is added after DKIM signing.

however we see no issues with hotmail DKIM signatures.
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Windows 2000: 640 MB ought to be enough for anybody

Re: Lots of FN because of VALIDITY* rules

[postgarage Graz IT]

Thanks for your help. I tried to reproduce the problem by reverting my 
changes to investigate it further with my newly learned knowledge, but 
now it works as intended, even when I get an "Excessive Queries" response.

IDK, perhaps the problem was something else and I "fixed" it by coincidence…

Anyway, thank you all.

On 6/3/24 14:52, Bill Cole wrote:

On 2024-06-03 at 08:35:32 UTC-0400 (Mon, 3 Jun 2024 14:35:32 +0200)
postgarage Graz IT 
is rumored to have said:

I think that the active.list file should be updated, when there are 
new rules, shouldn't it?


It is updated where it is actually used, on the ASF rule maintenance 
system. It is irrelevant to an operational deployment.


I have no idea why Debian installs that file at all.

Re: DKIM length 'l=' tag

[John Levine]

It appears that Bill Cole  said:
>Never has been safe. Terrible idea from the start. Never should have 
>been included in the specification.

Agreed.

>I was thinking of the same thing in a half-assed way, just catching 
>anything using the length tag. I'd bet that correlates to spam but we'd 
>need data to prove that.

When that blog post came out, some people I know at large providers
took a look at the DKIM signatures they were seeing. There was one ESP
that was signing their mail with l=1, but they stopped when we pointed
out what a bad idea that was. Some corporate systems that use Iroport
appliances are misconfigured to put l= with the actual body length.
I've been trying to track them down and encourage them to turn it off.

My advice is just to ignore the l= length. For the Irnport users, the
signature covers the entire body so it'll still validate. Other than
that I don't think it's a strong spam indicator but there's no reason
to try and guess whether a message with a length that doesn't cover the
full body has been modified maliciously.

R's,
John

Re: DKIM length 'l=' tag

[Bill Cole]

On 2024-06-03 at 07:05:29 UTC-0400 (Mon, 3 Jun 2024 12:05:29 +0100 
(BST))

Andrew C Aitchison 
is rumored to have said:


The DKIM RFC
   https://datatracker.ietf.org/doc/html/rfc6376#section-8.2
tells us that it is not safe to rely on the DKIM length (l=) tag


Never has been safe. Terrible idea from the start. Never should have 
been included in the specification.



and
   https://www.zone.eu/blog/2024/05/17/bimi-and-dmarc-cant-save-you/
shows how it can be used to subvert BIMI*.


I can't honestly say that I care. BIMI is a misguided concept useful 
only to marketers and the mythological creatures they call "consumers" 
who behave unlike many real humans.


I am looking at extending Mail::SpamAssassin::Plugin::DKIM to indicate 
when a DKIM body signature only covers part of the message body
and how much of the body is unsigned (bytes, percentage or possibly 
both).


I was thinking of the same thing in a half-assed way, just catching 
anything using the length tag. I'd bet that correlates to spam but we'd 
need data to prove that.


I am new to the spamassassin code, so any comments or suggetions would 
be welcome.


Resist the urge to refactor. It's easy to break things.


* I am not a fan of BIMI, but big name players appear to be using
it to display "trustable" logos on GUI mail clients, so users *will*
be caught when it breaks.


The concept that users should learn to trust logos as authentication per 
se is harmful. BIMI should be broken now and with every opportunity 
available. It is an indicator that a MUA author puts the interests of 
marketers ahead of the interests of users.



--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo@toad.social and many *@billmail.scconsult.com 
addresses)

Not Currently Available For Hire

Re: Lots of FN because of VALIDITY* rules

[Bill Cole]

On 2024-06-03 at 08:35:32 UTC-0400 (Mon, 3 Jun 2024 14:35:32 +0200)
postgarage Graz IT 
is rumored to have said:

I think that the active.list file should be updated, when there are 
new rules, shouldn't it?


It is updated where it is actually used, on the ASF rule maintenance 
system. It is irrelevant to an operational deployment.


I have no idea why Debian installs that file at all.

--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo@toad.social and many *@billmail.scconsult.com 
addresses)

Not Currently Available For Hire

Re: Lots of FN because of VALIDITY* rules

[Bill Cole]

On 2024-06-03 at 01:26:31 UTC-0400 (Mon, 3 Jun 2024 07:26:31 +0200)
postgarage Graz IT 
is rumored to have said:


Now for my questions:
*) as is stated in active.list it should not be edited. What's the 
correct place to add the new rules to activate them? local.cf?


Yes. In your local version of local.cf, typically in 
/etc/mail/spamassassin. This is as documented. Run "perldoc 
Mail::SpamAssassin::Conf" for the core configuration documentation.


Note that active.list is part of the rule management toolkit and IS NOT 
part of normal operations. It is part of the ruleset-building system 
that we use to create the daily update packages. In theory anyone could 
use that system to maintain rules and scores locally based on local 
data, as we do to produce daily updates, but I do not believe that 
anyone (literally *anyone*) other than the ASF does that.



*) If I understand it correctly
/var/lib/spamassassin/4.00/updates_spamassassin_org/ is updated by 
the SA update mechanism but it's the Linux distribution's 
responsibility to update /var/lib/spamassassin?


I'm not 100% clear on what that question means, perhaps because Debian 
does something different in /var/lib/spamassassin. The standard 
sa-update program will create versioned subdirectories under 
/var/lib/spamassassin/ as needed and channel subdirectories inside of 
them.



In that case should I fill a Debian bug? Or should the SA updates also 
include the file active.list?


SA updates include the active rules list in the form of the 72.active.cf 
file. The active.list file is not part of normal operations.



--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo@toad.social and many *@billmail.scconsult.com 
addresses)

Not Currently Available For Hire

Re: Lots of FN because of VALIDITY* rules

[postgarage Graz IT]

On 6/3/24 12:02, Matus UHLAR - fantomas wrote:
> On 03.06.24 07:26, postgarage Graz IT wrote:
>> A few days ago a lot of false negatives landed in our inboxes. As it
>> turned out the reason was that the for nearly all mails the
>> RCVD_IN_VALIDITY_CERTIFIED and RCVD_IN_VALIDITY_SAFE rules matched.
>>
>> I now know that validity introduced a query limit which we hit,
>> because I have to admit, I wasn't aware that I shouldn't use public
>> DNS resolvers for blacklists
>
> I'd say you should not use public DNS resolvers with mailserver.

Thanks. I know that by now and already set up a local DNS resolver. But 
that's not been the problem.


>> and therefore we got "Excessive Number of Queries" answers. I also
>> found this patch
>> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8244 which
>> introduces new rules addressing the query limit.
>
> my current rules show that all RCVD_IN_VALIDITY_* rules check for 
blocked.


The rules were here. The issue was, that they weren't active, as the 
were missing from the active.list file. There's no active.list under 
/var/lib/spamassassin/4.00/updates_spamassassin_org/. There's  only 
one in /usr/share/spamassassin/, which is provided by the debian 
package. But within that file the new *BLOCKED rules aren't activated.


So the situation was:
*) The updated *VALIDITY* rules were active
*) the new *VALIDITY*BLOCKED rules weren't active

Which lead to almost every mail passing the spamfilter, as for every 
"Excessive Number of Queries" answer from validity 
RCVD_IN_VALIDITY_CERTIFIED and RCVD_IN_VALIDITY_SAFE counted with -5 to 
the score.


AFTER I manually added the *BLOCKED rules to 
/usr/share/spamassassin/active.list I get the correct results
RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, 
RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001,


I think that the active.list file should be updated, when there are new 
rules, shouldn't it?



>> Those *BLOCKED rules where never applied because our spamassassin
>> received an updated rule-set which was saved to
>> /var/lib/spamassassin/4.00/updates_spamassassin_org/ but never
>> received an update for the active.list file located in
>> /usr/share/spamassassin/
>
>> After I manually added the changes from the above mentioned patch to
>> the active.list file it started to work.
>>
>> Now for my questions:
>> *) as is stated in active.list it should not be edited. What's the
>> correct place to add the new rules to activate them? local.cf?
>
> you can use dns_query_restriction to restrict which DNS lists to query.
>
> further, you can tune uridnsbl_skip_domain to avoid lookups for domains
> in URI* lists.
>
>> *) If I understand it correctly
>> /var/lib/spamassassin/4.00/updates_spamassassin_org/ is updated by
>> the SA update mechanism but it's the Linux distribution's
>> responsibility to update /var/lib/spamassassin? In that case should I

Sorry, that's a mistake by my side, should have been 
/usr/share/spamassassin/


>> fill a Debian bug? Or should the SA updates also include the file
>> active.list?
>
> reload spamd or amavis, the rules in /var/lib/spamassassin/ are used by
> default.
>
> Maybe you need to enable cron job by setting CRON=1 in
> /etc/default/spamassassin and it will happen automatically.
>
> ...I have no idea how active.list works.
>
>

RE: DKIM length 'l=' tag

[Marc]

> 
> 
> The DKIM RFC
> https://datatracker.ietf.org/doc/html/rfc6376#section-8.2
> tells us that it is not safe to rely on the DKIM length (l=) tag
> and
> https://www.zone.eu/blog/2024/05/17/bimi-and-dmarc-cant-save-you/
> shows how it can be used to subvert BIMI*.
> 
> I am looking at extending Mail::SpamAssassin::Plugin::DKIM to indicate
> when a DKIM body signature only covers part of the message body
> and how much of the body is unsigned (bytes, percentage or possibly
> both).
> 
> I am new to the spamassassin code, so any comments or suggetions would be
> welcome.
> 
> * I am not a fan of BIMI, but big name players appear to be using
> it to display "trustable" logos on GUI mail clients, so users *will*
> be caught when it breaks.
> 

Hi Andrew, this is a bit of topic, I posted this a while ago on the mailing 
list. But did you notice by any chance that eg. hotmail.com is failing every 
dkim verification (except their sender rewritten messages)?

DKIM length 'l=' tag

[Andrew C Aitchison]

The DKIM RFC
   https://datatracker.ietf.org/doc/html/rfc6376#section-8.2
tells us that it is not safe to rely on the DKIM length (l=) tag
and
   https://www.zone.eu/blog/2024/05/17/bimi-and-dmarc-cant-save-you/
shows how it can be used to subvert BIMI*.

I am looking at extending Mail::SpamAssassin::Plugin::DKIM to indicate 
when a DKIM body signature only covers part of the message body

and how much of the body is unsigned (bytes, percentage or possibly both).

I am new to the spamassassin code, so any comments or suggetions would be 
welcome.


* I am not a fan of BIMI, but big name players appear to be using
it to display "trustable" logos on GUI mail clients, so users *will*
be caught when it breaks.

Thanks,

--
Andrew C. Aitchison  Kendal, UK
   and...@aitchison.me.uk

Re: Lots of FN because of VALIDITY* rules

[Matus UHLAR - fantomas]

On 03.06.24 12:02, Matus UHLAR - fantomas wrote:

On 03.06.24 07:26, postgarage Graz IT wrote:
A few days ago a lot of false negatives landed in our inboxes. As it 
turned out the reason was that the for nearly all mails the 
RCVD_IN_VALIDITY_CERTIFIED and RCVD_IN_VALIDITY_SAFE rules matched.


I forgot to add that I have "lowered" (increased to small negative number) 
scores for RCVD_IN_VALIDITY_*, RCVD_IN_DNSWL_* and RCVD_IN_IADB_*

because I has similar bad experience with them.

I now know that validity introduced a query limit which we hit, 
because I have to admit, I wasn't aware that I shouldn't use public 
DNS resolvers for blacklists


I'd say you should not use public DNS resolvers with mailserver.

and therefore we got "Excessive Number of Queries" answers. I also 
found this patch 
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8244 which 
introduces new rules addressing the query limit.


my current rules show that all RCVD_IN_VALIDITY_* rules check for blocked.

Those *BLOCKED rules where never applied because our spamassassin 
received an updated rule-set which was saved to 
/var/lib/spamassassin/4.00/updates_spamassassin_org/ but never 
received an update for the active.list file located in 
/usr/share/spamassassin/


After I manually added the changes from the above mentioned patch to 
the active.list file it started to work.


Now for my questions:
*) as is stated in active.list it should not be edited. What's the 
correct place to add the new rules to activate them? local.cf?


you can use dns_query_restriction to restrict which DNS lists to query.

further, you can tune uridnsbl_skip_domain to avoid lookups for 
domains in URI* lists.



*) If I understand it correctly
/var/lib/spamassassin/4.00/updates_spamassassin_org/ is updated 
by the SA update mechanism but it's the Linux distribution's 
responsibility to update /var/lib/spamassassin? In that case should 
I fill a Debian bug? Or should the SA updates also include the file 
active.list?


reload spamd or amavis, the rules in /var/lib/spamassassin/ are used 
by default.


Maybe you need to enable cron job by setting CRON=1 in 
/etc/default/spamassassin and it will happen automatically.


...I have no idea how active.list works.


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Enter any 12-digit prime number to continue.

Re: Lots of FN because of VALIDITY* rules

[Matus UHLAR - fantomas]

On 03.06.24 07:26, postgarage Graz IT wrote:
A few days ago a lot of false negatives landed in our inboxes. As it 
turned out the reason was that the for nearly all mails the 
RCVD_IN_VALIDITY_CERTIFIED and RCVD_IN_VALIDITY_SAFE rules matched.


I now know that validity introduced a query limit which we hit, 
because I have to admit, I wasn't aware that I shouldn't use public 
DNS resolvers for blacklists


I'd say you should not use public DNS resolvers with mailserver.

and therefore we got "Excessive Number of 
Queries" answers. I also found this patch 
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8244 which 
introduces new rules addressing the query limit.


my current rules show that all RCVD_IN_VALIDITY_* rules check for blocked.

Those *BLOCKED rules where never applied because our spamassassin 
received an updated rule-set which was saved to 
/var/lib/spamassassin/4.00/updates_spamassassin_org/ but never 
received an update for the active.list file located in 
/usr/share/spamassassin/


After I manually added the changes from the above mentioned patch to 
the active.list file it started to work.


Now for my questions:
*) as is stated in active.list it should not be edited. What's the 
correct place to add the new rules to activate them? local.cf?


you can use dns_query_restriction to restrict which DNS lists to query.

further, you can tune uridnsbl_skip_domain to avoid lookups for domains in 
URI* lists.



*) If I understand it correctly
/var/lib/spamassassin/4.00/updates_spamassassin_org/ is updated by 
the SA update mechanism but it's the Linux distribution's 
responsibility to update /var/lib/spamassassin? In that case should I 
fill a Debian bug? Or should the SA updates also include the file 
active.list?


reload spamd or amavis, the rules in /var/lib/spamassassin/ are used by 
default.


Maybe you need to enable cron job by setting CRON=1 in 
/etc/default/spamassassin and it will happen automatically.


...I have no idea how active.list works.


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety. -- Benjamin Franklin, 1759

Re: TxRep does not evaluate EMAIL_IP reputation

[giovanni]

On 6/3/24 1:10 AM, Tomohiro Hosaka wrote:

Slight correction.

2024-06-03 07:55 に Tomohiro Hosaka さんは書きました:

Here $rc is dualvar.
https://metacpan.org/pod/DBI#execute


This is not dualvar, exactly.

However, the patch is unchanged.
Evaluated as a bool, it is "0E0" true; evaluated as a number, it is the number 
of cases.
You may use $cnt for more simplicity.

Hi,
could you please open bug reports on https://bz.apache.org/SpamAssassin/ so 
that we can track them ?
 Thanks
   Giovanni


OpenPGP_signature.asc
Description: OpenPGP digital signature

Lots of FN because of VALIDITY* rules

[postgarage Graz IT]

Hello!

Debian 12.5
SpamAssassin version 4.0.0
  running on Perl version 5.36.0

Server setup with iRedMail


A few days ago a lot of false negatives landed in our inboxes. As it 
turned out the reason was that the for nearly all mails the 
RCVD_IN_VALIDITY_CERTIFIED and RCVD_IN_VALIDITY_SAFE rules matched.


I now know that validity introduced a query limit which we hit, because 
I have to admit, I wasn't aware that I shouldn't use public DNS 
resolvers for blacklists and therefore we got "Excessive Number of 
Queries" answers. I also found this patch 
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8244 which introduces 
new rules addressing the query limit.


Those *BLOCKED rules where never applied because our spamassassin 
received an updated rule-set which was saved to 
/var/lib/spamassassin/4.00/updates_spamassassin_org/ but never 
received an update for the active.list file located in 
/usr/share/spamassassin/
After I manually added the changes from the above mentioned patch to the 
active.list file it started to work.


Now for my questions:
*) as is stated in active.list it should not be edited. What's the 
correct place to add the new rules to activate them? local.cf?

*) If I understand it correctly
/var/lib/spamassassin/4.00/updates_spamassassin_org/ is updated by 
the SA update mechanism but it's the Linux distribution's responsibility 
to update /var/lib/spamassassin? In that case should I fill a Debian 
bug? Or should the SA updates also include the file active.list?


Thanks and best regards
Flo

Re: TxRep does not evaluate EMAIL_IP reputation

[Tomohiro Hosaka]

Slight correction.

2024-06-03 07:55 に Tomohiro Hosaka さんは書きました:

Here $rc is dualvar.
https://metacpan.org/pod/DBI#execute


This is not dualvar, exactly.

However, the patch is unchanged.
Evaluated as a bool, it is "0E0" true; evaluated as a number, it is the 
number of cases.

You may use $cnt for more simplicity.

TxRep does not evaluate EMAIL_IP reputation

[Tomohiro Hosaka]

Hello.

EMAIL_IP is not evaluated with SQLBasedAddrList.

In conclusion, the following patches are needed.

--- 
../Mail-SpamAssassin-4.0.1.orig/lib/Mail/SpamAssassin/SQLBasedAddrList.pm	2024-03-26 
13:52:11.0 +0900
+++ 
../Mail-SpamAssassin-4.0.1/lib/Mail/SpamAssassin/SQLBasedAddrList.pm	2024-06-03 
07:39:19.620985000 +0900

@@ -190,79 +190,79 @@
 sub get_addr_entry {
   my ($self, $addr, $signedby) = @_;

   my $entry = { addr => $addr,
 exists_p => 0,
 msgcount=> 0,
 totscore => 0,
 signedby => $signedby,
   };

   my ($email, $ip) = $self->_unpack_addr($addr);

   return $entry  unless $email ne '' && (defined $ip || defined 
$signedby);


   my $sql;
   my $sth;
   my $rc;
   if($self->{main}->{conf}->{txrep_welcomelist_out} and ($email =~ 
/(?:[^\s\@]+)\@(?:[^\s\@]+)/)) {

 $sql = "SELECT msgcount, totscore FROM $self->{tablename} " .
 "WHERE username = ? AND email = ? AND ip = 
'WELCOMELIST_OUT'";

 $sth = $self->{dbh}->prepare($sql);
 unless (defined($sth)) {
   info("auto-welcomelist: sql-based get_addr_entry %s: SQL prepare 
error: %s",

  "$self->{_username}|$ip", $self->{dbh}->errstr);
 }
 $rc = $sth->execute($self->{_username}, $email);
 my $cnt = 0;
 my $aryref;
 # how to combine data if there are several entries (like signed by
 # an author domain and by a remailer)?  for now just take an 
average

 while ( defined($aryref = $sth->fetchrow_arrayref()) ) {
   if (defined $entry->{msgcount} && defined $aryref->[1]) {
 $entry->{msgcount} = $aryref->[0];
 $entry->{totscore} = $aryref->[1];
   }
   $entry->{exists_p} = 1;
   $cnt++;
 }
 $sth->finish();
-return $entry if $rc;
+return $entry if $rc>=1;
   }
   undef $sth;
   undef $rc;

   $sql = "SELECT msgcount, totscore FROM $self->{tablename} " .
 "WHERE username = ? AND email = ?";


Here $rc is dualvar.
https://metacpan.org/pod/DBI#execute

'if $rc' will be true even if the number of cases is zero.

I noticed this while doing $sa->check in the test environment.

*  3.2 TXREP TXREP: Score normalizing based on sender's reputation
*  [EMAILIP: u...@host.com, rep: 24.34, count: 1] <--- This line 
will not appear without patch above.

*  [DOMAIN: host.com, rep: 4.87, count: 32]
*  [IP: 123.123.123.123, rep: 6.35, count: 34]


if($self->{main}->{conf}->{txrep_welcomelist_out} and ($email =~ 
/(?:[^\s\@]+)\@(?:[^\s\@]+)/)) {

Entering the if branch above will always return $entry.
This seems to be a pretty big bug.

Thanks.

TxRep may increase false positives

[Tomohiro Hosaka]

Hello.

I am using TxRep with DBBasedAddrList.

If we learn the following email
ham is email address user@host with signed
spam is email address without user@host without signed

The following reputation is used
ham is  [EMAILIP: user@host, rep:xx, count: xx]
spam is [EMAIL: user@host, rep: xx, count: xx].

Because the same storage location is used, the data will be mixed up.
If more spam is learned, there is a risk of false positives for ham.
This is because the weight of EMAILIP is particularly high.

The reason is that $signedby is not used in 
DBBasedAddrList::get_addr_entry.


SQLBasedAddrList::get_addr_entry uses $signedby.
Here is a quote from the auto_welcomelist_distinguish_signed section of 
TxRep.pm.
Without this option, or for domains that do not use a DKIM signature, 
the reputation of legitimate email can get mixed with the reputation of 
forgeries.


Given the above statement, I assume the developer knows this.

If the current REPUTATION LOGICS are to be retained, then
DBBasedAddrList::get_addr_entry should use $signedby as well as 
SQLBasedAddrList::get_addr_entry?


Thanks.

TxRep's learn and forget are not contrasted

[Tomohiro Hosaka]

Hello.

In conclusion, the following patch is needed.


--- 
../Mail-SpamAssassin-4.0.1.orig/lib/Mail/SpamAssassin/Plugin/TxRep.pm
   2024-03-26 13:52:09.0 +0900
+++ ../Mail-SpamAssassin-4.0.1/lib/Mail/SpamAssassin/Plugin/TxRep.pm
2024-06-01 05:09:01.496565000 +0900

@@ -1967,10 +1967,13 @@
 sub forget_message {
 
###

   my ($self, $params) = @_;
   return 0 unless ($self->{conf}->{use_txrep});
   my $pms = ($self->{last_pms})? $self->{last_pms} : 
Mail::SpamAssassin::PerMsgStatus->new($self->{main}, $params->{msg});
+  if (!defined $pms->{relays_internal} && !defined 
$pms->{relays_external}) {

+$pms->extract_message_metadata();
+  }

   dbg("TxRep: forgetting a message");
   $self->{forgetting} = 1;
   my $ret = $self->check_senders_reputation($pms);
   $self->{forgetting} = undef;


The three additional lines are a copy of learn_message immediately 
above.

Probably unnecessary if forget_message is called from line 1349.
However, it is necessary when called as $sa->learn($msg, $id=undef, 
$isspam=1, $forget=1) as in sa-learn.



Below is a demonstration.

abstract code
$sa->learn($msg, $id=undef, $isspam=1, $forget=0);
dump_tx_reputation();
$sa->learn($msg, $id=undef, $isspam=1, $forget=1);
dump_tx_reputation();

Without patch
---
123.123.123.123|ip=none: 1
123.123.123.123|ip=none|totscore: 20
6e4a149a9841940055d7dc4e85c0c750aef09cfb@sa_generated|ip=none: 2

6e4a149a9841940055d7dc4e85c0c750aef09cfb@sa_generated|ip=none|totscore: 
40

host.name|ip=123.123: 1
host.name|ip=123.123|totscore: 20
u...@host.name|ip=123.123: 1
u...@host.name|ip=123.123|totscore: 20
u...@host.name|ip=none: 1
u...@host.name|ip=none|totscore: 20
---
123.123.123.123|ip=none: 1
123.123.123.123|ip=none|totscore: 20
host.name|ip=123.123: 1
host.name|ip=123.123|totscore: 20
host.name|ip=none: 1  <-
host.name|ip=none|totscore: -20   <-
u...@host.name|ip=123.123: 1
u...@host.name|ip=123.123|totscore: 20
u...@host.name|ip=none: 1
u...@host.name|ip=none|totscore: 0

With patch
---
123.123.123.123|ip=none: 1
123.123.123.123|ip=none|totscore: 20
6e4a149a9841940055d7dc4e85c0c750aef09cfb@sa_generated|ip=none: 2

6e4a149a9841940055d7dc4e85c0c750aef09cfb@sa_generated|ip=none|totscore: 
40

host.name|ip=123.123: 1
host.name|ip=123.123|totscore: 20
u...@host.name|ip=123.123: 1
u...@host.name|ip=123.123|totscore: 20
u...@host.name|ip=none: 1
u...@host.name|ip=none|totscore: 20
---
123.123.123.123|ip=none: 1
123.123.123.123|ip=none|totscore: 0
host.name|ip=123.123: 1
host.name|ip=123.123|totscore: 0
u...@host.name|ip=123.123: 1
u...@host.name|ip=123.123|totscore: 0
u...@host.name|ip=none: 1
u...@host.name|ip=none|totscore: 0


Thanks.

Re: TxRep does not read setting|default value

[Tomohiro Hosaka]

Hello.

Thanks for the reply.

Added txrep_dilution_factor 0.98 to 
/usr/local/etc/mail/spamassassassin/local.cf


 340   push (@cmds, {
 341 setting => 'txrep_dilution_factor',
 342 default => 0.98,
 343 type=> $Mail::SpamAssassin::Conf::CONF_TYPE_NUMERIC,
 344 code=> sub {
 345 my ($self, $key, $value, $line) = @_;
 346 if ($value < 0.7 || $value > 1.0) {return 
$Mail::SpamAssassin::Conf::INVALID_VALUE;}

 347 warn $self;
 348 use Carp qw(cluck);
 349 cluck "This is how we got here!";
 350 $self->{txrep_dilution_factor} = $value;
 351 }
 352   });


Mail::SpamAssassin::Conf=HASH(0x8577b09d8) at 
/usr/local/lib/perl5/site_perl/IO/All.pm line 148.
This is how we got here! at 
/usr/local/lib/perl5/site_perl/Mail/SpamAssassin/Plugin/TxRep.pm line 
349.

Mail::SpamAssassin::Plugin::TxRep::__ANON__(Mail::SpamAssassin::Conf=HASH(0x8577b09d8), 
"txrep_dilution_factor", 0.98, "txrep_dilution_factor 0.98") called at 
/usr/local/lib/perl5/site_perl/Mail/SpamAssassin/Conf/Parser.pm line 437

Mail::SpamAssassin::Conf::Parser::parse(Mail::SpamAssassin::Conf::Parser=HASH(0x84e63dbe8), 
"file start (pre_config_text)\x{a}\x{a}file end 
(pre_config_text)\x{a}file"..., 0) called at 
/usr/local/lib/perl5/site_perl/Mail/SpamAssassin/Conf.pm line 5033

Mail::SpamAssassin::Conf::parse_rules(Mail::SpamAssassin::Conf=HASH(0x8577b09d8), 
"file start (pre_config_text)\x{a}\x{a}file end 
(pre_config_text)\x{a}file"...) called at 
/usr/local/lib/perl5/site_perl/Mail/SpamAssassin.pm line 1792
Mail::SpamAssassin::init(Mail::SpamAssassin=HASH(0x8577d42a0), 
1) called at /usr/local/lib/perl5/site_perl/Mail/SpamAssassin.pm line 
709

Mail::SpamAssassin::init_learner(Mail::SpamAssassin=HASH(0x8577d42a0), 
HASH(0x84e63d930)) called at /root/myops/script/sa_scan.pl line 336

main::new_sa() called at /root/myops/script/sa_scan.pl line 145
main::run(main=HASH(0x82f4515b8)) called at 
/root/myops/script/sa_scan.pl line 386


$self is Mail::SpamAssassin::Conf=HASH(0x8577b09d8) here.
It is not Mail::SpamAssassin::Plugin::TxRep=HASH(...).

I tried the bottom and it was the same.

--- 
/usr/local/lib/perl5/site_perl/Mail/SpamAssassin/Plugin/TxRep.pm.orig
   2024-05-30 22:22:55.412407000 +0900
+++ /usr/local/lib/perl5/site_perl/Mail/SpamAssassin/Plugin/TxRep.pm
2024-05-30 22:23:37.990091000 +0900

@@ -1675,6 +1675,7 @@

   # performing the dilution aging correction
   if (defined $self->total() && defined $self->count() && 
$self->count() > 0 && defined $self->{txrep_dilution_factor}) {

+die; # It will never be processed here.
 my $diluted_total =
 ($self->count() + 1) *
 ($self->{txrep_dilution_factor} * $self->total() + $score) /

Thanks.

2024-05-30 21:38 に Bill Cole さんは書きました:

On 2024-05-30 at 03:58:18 UTC-0400 (Thu, 30 May 2024 16:58:18 +0900)
Tomohiro Hosaka 
is rumored to have said:


Hello.

The code seems to be wrong.


I do not believe that to be so. See lines 340-347 in TxRep.pm.

Re: TxRep does not read setting|default value

[Bill Cole]

On 2024-05-30 at 03:58:18 UTC-0400 (Thu, 30 May 2024 16:58:18 +0900)
Tomohiro Hosaka 
is rumored to have said:

> Hello.
>
> The code seems to be wrong.

I do not believe that to be so. See lines 340-347 in TxRep.pm.


-- 
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo@toad.social and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

TxRep does not read setting|default value

[Tomohiro Hosaka]

Hello.

The code seems to be wrong.

Thanks.



Mail-SpamAssassin-4.0.1

--- lib/Mail/SpamAssassin/Plugin/TxRep.pm.orig  2024-03-26 
13:52:09.0 +0900
+++ lib/Mail/SpamAssassin/Plugin/TxRep.pm   2024-05-30 
16:50:22.708673000 +0900

@@ -1668,11 +1668,11 @@
   $self->{entry}->{msgcount} ||= 0;

   # performing the dilution aging correction
-  if (defined $self->total() && defined $self->count() && 
$self->count() > 0 && defined $self->{txrep_dilution_factor}) {
+  if (defined $self->total() && defined $self->count() && 
$self->count() > 0 && defined $self->{conf}{txrep_dilution_factor}) {

 my $diluted_total =
 ($self->count() + 1) *
-($self->{txrep_dilution_factor} * $self->total() + $score) /
-($self->{txrep_dilution_factor} * $self->count() + 1);
+($self->{conf}{txrep_dilution_factor} * $self->total() + 
$score) /

+($self->{conf}{txrep_dilution_factor} * $self->count() + 1);
 my $corrected_score = $diluted_total - $self->total();
 $self->{checker}->add_score($self->{entry}, $corrected_score);
   } else {

Re: How to report SPAM?

[Frido Otten]

They do if you're offering mail service to a large number of users. They 
login to a phished mailbox, send new phishingmails to that mailbox and 
check the headers if they can see which rules are hit. Then they adapt 
the phishingmail to get a lower score until they are below the spam 
threshold. That's why I am writing my own rules with very generic names 
and description.


Op 27-05-2024 om 23:10 schreef Thomas Barth via users:
What can I do? With these SPAMS, I have the impression that the 
senders know exactly how to trick Spamassassin.


OpenPGP_0xCCDCFB22C59E9DD2.asc
Description: OpenPGP public key


OpenPGP_signature.asc
Description: OpenPGP digital signature

body rule only for txt or html?

[Tobi]

Hello list

ifplugin Mail::SpamAssassin::Plugin::ExtractText
extracttext_external  pdfgrep  /usr/bin/pdfgrep .+ {}
extracttext_use   pdfgrep  .pdf application/pdf
endif

which leads to the fact that body rules then can also hit on pdf
content. Is there a possibility for a rule to search only in the text
and/or html representation of a message?

Cheers

tobi

RE: dkim fail %

[Marc]

> > I am only looking at signature verifications of dkim, nothing else.  My
> > software currently does not log selector and domain of failing
> signatures,
> > so I am just doing an mx lookup and 'guessing' that outgoing mail
> > originate from something similar.  It is just to much of a coincidence
> > that everything is outlook.  Maybe my software or their software is not
> > 100% compatible with what is being signed.
> 
> what about replacing such software? With one that logs proper info?
> 

Not so relevant at this stage. If I grep everything from hotmail.com than the 
only ones that do not fail my dkim are the ones that have sender rewrite 
applied to by hotmail.com.

So does anyone else have this that x...@hotmail.com always fails (except for 
srs mails)?

Re: dkim fail %

[Matus UHLAR - fantomas]

> I am having a large (20%) of messages fail dkim. If I do some random
> checks, it looks like most of the failing messages are from the
> outlook.com cloud. Does any one else have this? Or is my setup just not
> properly checking dkim of outlook.com?



how should i guess ?

i see o365 not dkim sign at all, this is ok when spf pass, but not often
spf_helo fails, should dmarc care of spf_helo ? :)

more help needs more info from you


On 28.05.24 12:47, Marc wrote:
I am only looking at signature verifications of dkim, nothing else.  My 
software currently does not log selector and domain of failing signatures, 
so I am just doing an mx lookup and 'guessing' that outgoing mail 
originate from something similar.  It is just to much of a coincidence 
that everything is outlook.  Maybe my software or their software is not 
100% compatible with what is being signed.


what about replacing such software? With one that logs proper info?


add: header: X-Verification-Result: dkim=fail  -@ xxx...@karllagerfeld.com
[@]# dig +short -t mx karllagerfeld.com
10 fallback1.mx.nxs.nl.
5 karllagerfeld-com.mail.protection.outlook.com.


add: header: X-Verification-Result: dkim=fail -@ ...@hotmail.com
[@]# dig +short -t mx hotmail.com
2 hotmail-com.olc.protection.outlook.com.


etc etc.


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
(R)etry, (A)bort, (C)ancer

RE: dkim fail %

[Marc]

> > I am having a large (20%) of messages fail dkim. If I do some random
> > checks, it looks like most of the failing messages are from the
> > outlook.com cloud. Does any one else have this? Or is my setup just not
> > properly checking dkim of outlook.com?
> 
> how should i guess ?
> 
> i see o365 not dkim sign at all, this is ok when spf pass, but not often
> spf_helo fails, should dmarc care of spf_helo ? :)
> 
> more help needs more info from you

I am only looking at signature verifications of dkim, nothing else. My software 
currently does not log selector and domain of failing signatures, so I am just 
doing an mx lookup and 'guessing' that outgoing mail originate from something 
similar. It is just to much of a coincidence that everything is outlook. Maybe 
my software or their software is not 100% compatible with what is being signed.

add: header: X-Verification-Result: dkim=fail  -@ xxx...@karllagerfeld.com
[@]# dig +short -t mx karllagerfeld.com
10 fallback1.mx.nxs.nl.
5 karllagerfeld-com.mail.protection.outlook.com.


add: header: X-Verification-Result: dkim=fail -@ ...@hotmail.com
[@]# dig +short -t mx hotmail.com
2 hotmail-com.olc.protection.outlook.com.


etc etc.

Re: dkim fail %

[Benny Pedersen]

Marc skrev den 2024-05-28 14:15:
I am having a large (20%) of messages fail dkim. If I do some random 
checks, it looks like most of the failing messages are from the 
outlook.com cloud. Does any one else have this? Or is my setup just not 
properly checking dkim of outlook.com?


how should i guess ?

i see o365 not dkim sign at all, this is ok when spf pass, but not often 
spf_helo fails, should dmarc care of spf_helo ? :)


more help needs more info from you

dkim fail %

[Marc]

I am having a large (20%) of messages fail dkim. If I do some random checks, it 
looks like most of the failing messages are from the outlook.com cloud. Does 
any one else have this? Or is my setup just not properly checking dkim of 
outlook.com?

Re: How to report SPAM?

[Matus UHLAR - fantomas]

On 27.05.24 23:10, Thomas Barth via users wrote:
for months I have been waiting for the type of SPAM I receive to be 
captured by the DNS block lists. But nothing is happening. I have long 
since fed Spamassassin with these SPAMs. What else can I do? I have 
even activated HOSTKARMA-black/brown. Doesn't help either. Do I 
perhaps have to report the SPAM myself? Is this reporting still up to 
date 
https://cwiki.apache.org/confluence/display/SPAMASSASSIN/Report+spam




The scoring of this type of SPAM is
X-Spam-Status: No, score=3.502 tagged_above=2 required=6.31
   tests=[BAYES_99=3.5, BAYES_999=0.2, DKIM_SIGNED=0.1, 
DKIM_VALID=-0.1,

   DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, DMARC_PASS=-0.001,
   HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_BL=0.001, 
RCVD_IN_MSPIKE_L3=0.001,
   SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=no 
autolearn_force=no


From the score itself it's very hard to find out the issue.
Maybe you are blocked on DNS blocklist (perhaps you use public DNS 
servers)? Perhaps the spam came from hosts that are not blocked?


If you posted Received: headers (here or on e.g. pastebin), it could help us.


Here the checks of a higher rated SPAM mail. A lot more working checks 
available.


X-Spam-Status: Yes, score=15.037 tagged_above=2 required=6.31
   tests=[BAYES_20=-0.001, DMARC_MISSING=0.001, EXTRA_SCORE=1,
   FROM_SUSPICIOUS_NTLD=0.499, FROM_SUSPICIOUS_NTLD_FP=1.999,
   FSL_BULK_SIG=0.001, HTML_FONT_LOW_CONTRAST=0.001, 
HTML_IMAGE_RATIO_04=0.001,

   HTML_MESSAGE=0.001, MIME_HTML_ONLY=0.1, MISSING_MID=0.497,
   NORDNS_LOW_CONTRAST=0.001, RAZOR2_CF_RANGE_51_100=1.886, 
RAZOR2_CHECK=0.922,
   RCVD_IN_HOSTKARMA_BL=2, RCVD_IN_MSPIKE_BL=0.001, 
RCVD_IN_MSPIKE_ZBI=0.001,

   RCVD_IN_SBL_CSS=3.335, RDNS_NONE=0.793, RELAYCOUNTRY_BAD=2,
   SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, 
TO_NO_BRKTS_NORDNS_HTML=0.001]

   autolearn=no autolearn_force=no


So, at least dnsbls work well for you.

What can I do? With these SPAMS, I have the impression that the 
senders know exactly how to trick Spamassassin.


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
You have the right to remain silent. Anything you say will be misquoted,
then used against you.

Re: "deadline shrunk" in logs ?

[Bill Cole]

On 2024-05-27 at 17:43:43 UTC-0400 (Mon, 27 May 2024 17:43:43 -0400)
J Doe 
is rumored to have said:

> Hi list,
>
> Sometimes when I am checking my e-mail server logs, SA will note
> "deadline shrunk":
>
> May 27 12:56:07 server spamd[29305]: async: aborting after 4.253 s,
> deadline shrunk: DNSBL, A/106.55.47.104.dnsbl.sorbs.net, rules:
> RCVD_IN_SORBS_DUL, __RCVD_IN_SORBS
>
> What does the expression "deadline shrunk" mean ?


It means that for some reason, the abort_remaining_lookups() function was 
called before all pending DNS queries were complete and before the fixed 
timeout deadline was reached. The most common cause is a DNS-based rule 
configured to shortcircuit while other queries are outstanding.

-- 
Bill Cole

"deadline shrunk" in logs ?

[J Doe]

Hi list,

Sometimes when I am checking my e-mail server logs, SA will note
"deadline shrunk":

May 27 12:56:07 server spamd[29305]: async: aborting after 4.253 s,
deadline shrunk: DNSBL, A/106.55.47.104.dnsbl.sorbs.net, rules:
RCVD_IN_SORBS_DUL, __RCVD_IN_SORBS

What does the expression "deadline shrunk" mean ?

Thanks,

- J

RE: How to report SPAM?

[Marc]

> for months I have been waiting for the type of SPAM I receive to be
> captured by the DNS block lists. But nothing is happening. I have long
> since fed Spamassassin with these SPAMs. What else can I do?

put your spam score lower? I don't think you will get many false positives when 
you put it at 3

> I have even
> activated HOSTKARMA-black/brown. Doesn't help either. Do I perhaps have
> to report the SPAM myself? 

I started creating own dns blacklists. I am flagging a lot as spam and users 
can individually unset this.

Re: kam fails if askdns is disabled

[Benny Pedersen]

Noel Butler skrev den 2024-05-26 01:53:


Shame on you for not turning on ESP  ;)


whois Kevin ? :)


When Benny is off his meds, he's like the newbies who lodge support
tickets saying  "mail doesnt work"  not I cant get my mail because of
error fooXXX or cant send mail because im an idiot and cant read that
we dont relay out on port 25, or im trying to relay using my old isps
mail server... *sigh*  but you get used to ignoring Benny's
unintelligible shit.


i can still add you to sieve autoreader, so i dont need to reply

Re: kam fails if askdns is disabled

[Noel Butler]

On 26/05/2024 01:20, Antony Stone wrote:


On Saturday 25 May 2024 at 16:57:21, Benny Pedersen wrote:

Antony Stone skrev den 2024-05-25 16:52: Is this a reply to something?
something ?, try disable askdns plugin, then do spamassassin --lint

succes ?

hopefully kam know why

there should not be lint errors if just check plugin is enabled, where
all other plugins is disabled


I apologise for not having worked that out from "+1".

Antony.

Shame on you for not turning on ESP  ;)

When Benny is off his meds, he's like the newbies who lodge support 
tickets saying  "mail doesnt work"  not I cant get my mail because of 
error fooXXX or cant send mail because im an idiot and cant read that we 
dont relay out on port 25, or im trying to relay using my old isps mail 
server... *sigh*  but you get used to ignoring Benny's unintelligible 
shit.


--
Regards,
Noel Butler

Re: kam fails if askdns is disabled

[Antony Stone]

On Saturday 25 May 2024 at 16:57:21, Benny Pedersen wrote:

> Antony Stone skrev den 2024-05-25 16:52:
> > Is this a reply to something?
> 
> something ?, try disable askdns plugin, then do spamassassin --lint
> 
> succes ?
> 
> hopefully kam know why
> 
> there should not be lint errors if just check plugin is enabled, where
> all other plugins is disabled

I apologise for not having worked that out from "+1".

Antony.

-- 
I lay awake all night wondering where the sun went, and then it dawned on me.

   Please reply to the list;
 please *don't* CC me.

Re: kam fails if askdns is disabled

[Benny Pedersen]

Antony Stone skrev den 2024-05-25 16:52:


Is this a reply to something?


something ?, try disable askdns plugin, then do spamassassin --lint

succes ?

hopefully kam know why

there should not be lint errors if just check plugin is enabled, where 
all other plugins is disabled

Re: kam fails if askdns is disabled

[Antony Stone]

On Saturday 25 May 2024 at 16:51:07, Benny Pedersen wrote:

> +1

Is this a reply to something?

Antony.

-- 
"Linux is going to be part of the future. It's going to be like Unix was."

 - Peter Moore, Asia-Pacific general manager, Microsoft

   Please reply to the list;
 please *don't* CC me.

kam fails if askdns is disabled

[Benny Pedersen]

+1

Re: shared lock, exclusive lock (bayes_seen,bayes_toks,tx-reputation)

[Benny Pedersen]

Tomohiro Hosaka skrev den 2024-05-25 13:43:


Perhaps SpamAssassin is designed for single-process use?


this is a limit on DB_File only

(If so, this would conflict with the preforked spamd, which does not 
seem to have any special locking to prevent this on the spamd side.)


spamd only write in one pid process, it can still read in multi pids, 
think what should happen if DB_File could do more then one write at a 
time ?



I would be very grateful if someone could give me an answer.


choise another store for bayes solves it, maybe radical change to redis 
? :)


i just use postgres


Thank you in advance.


+1

shared lock, exclusive lock (bayes_seen,bayes_toks,tx-reputation)

[Tomohiro Hosaka]

Hello.

I have a question about Mail::SpamAssassin::BayesStore::DBM (DB_File).
I am using it with Mail::SpamAssassin::Locker::Flock.

I think this module is implemented as follows
For reading, tie_db_readonly tie (no lock)
For writing, tie_db_writable flock LOCK_EX & tie

multi-process $sa->check or $sa->learn
or spamd prefork if started
Is there any problem with spamassassin not doing flock LOCK_SH for 
reads?

Am I missing something in my code reading?

I did a stress experiment on DB_File at hand.
Without flock LOCK_SH for reads, the process does not terminate 
abnormally and continues to run, but sometimes empty data is returned or 
data with half-key is returned.
From this experiment, it seems that flock LOCK_SH is indeed necessary 
for reads.


https://metacpan.org/pod/DB_File#Locking:-The-Trouble-with-fd
The above also seems to assume that locking is required for reads as 
well.


Perhaps SpamAssassin is designed for single-process use?
(If so, this would conflict with the preforked spamd, which does not 
seem to have any special locking to prevent this on the spamd side.)


I would be very grateful if someone could give me an answer.

Thank you in advance.

Re: Extract Local-part from To: Adress to use in spamassassin rule

[giovanni]

On 5/23/24 5:39 PM, Bill Cole wrote:

On 2024-05-23 at 03:40:48 UTC-0400 (Thu, 23 May 2024 09:40:48 +0200)
Carsten 
is rumored to have said:


Hi @all,

I want to create a SpamAssassin rule that checks if the subject line of an email contains the local 
part of the recipient's email address (the part before the @ symbol). For example, if the 
recipient's email address is |i...@example.com|, I want to check if the subject contains the phrase 
"info lorem ipsum". If the recipient's email address is |foo...@example.com|, I want to 
check if the subject contains the phrase "foobar lorem ipsum". The rule should be general 
and adaptable to different local parts of email addresses.

*Requirements:*

1. Extract the local part of the recipient's email address from the
   |To| header.
2. Use the extracted local part to check if it is present in the
   |Subject| header.
3. The rule should be written in a way that works for any local part of
   the email address, not just a specific one.


See the section titled "CAPTURING TAGS USING REGEX NAMED CAPTURE GROUPS" in the 
embedded configuration documentation (perldoc Mail::SpamAssassin::Conf) for how to 
capture a pattern in one rule and use it in another. I don't have a working rule for you, 
but that's the mechanism I would use.


If you need same samples to start with, take a look at 
https://github.com/apache/spamassassin/blob/094428cf11b0ad8d5658fd18d62d69663357fb10/rulesrc/sandbox/gbechis/20_misc.cf#L98

  Giovanni



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: Extract Local-part from To: Adress to use in spamassassin rule

[Jimmy]

Hi,

Try this

if (version >= 4.00)
 if can(Mail::SpamAssassin::Conf::feature_capture_rules)
   header   __TZ_CAP_TO_USR   To:addr =~ /(?[^@]+)/
   header   __TZ_SUBJ_HAS_USR Subject =~ /\b%{TZ_TO_USR}\b/i
 endif
endif

I'm curious if CAPTURING TAGS can handle multiple groups within the same
rules like this?

header   __TZ_CAP_TO_ADDR   To:addr =~
/(?)\@(?)/

Jimmy




On Thu, May 23, 2024 at 2:40 PM Carsten  wrote:

> Hi @all,
>
> I want to create a SpamAssassin rule that checks if the subject line of an
> email contains the local part of the recipient's email address (the part
> before the @ symbol). For example, if the recipient's email address is
> i...@example.com, I want to check if the subject contains the phrase
> "info lorem ipsum". If the recipient's email address is foo...@example.com,
> I want to check if the subject contains the phrase "foobar lorem ipsum".
> The rule should be general and adaptable to different local parts of email
> addresses.
>
> *Requirements:*
>
>1. Extract the local part of the recipient's email address from the To
>header.
>2. Use the extracted local part to check if it is present in the
>Subject header.
>3. The rule should be written in a way that works for any local part
>of the email address, not just a specific one.
>
> Thank you very much for your suggestions
>

Re: Extract Local-part from To: Adress to use in spamassassin rule

[Bill Cole]

On 2024-05-23 at 03:40:48 UTC-0400 (Thu, 23 May 2024 09:40:48 +0200)
Carsten 
is rumored to have said:


Hi @all,

I want to create a SpamAssassin rule that checks if the subject line 
of an email contains the local part of the recipient's email address 
(the part before the @ symbol). For example, if the recipient's email 
address is |i...@example.com|, I want to check if the subject contains 
the phrase "info lorem ipsum". If the recipient's email address is 
|foo...@example.com|, I want to check if the subject contains the 
phrase "foobar lorem ipsum". The rule should be general and adaptable 
to different local parts of email addresses.


*Requirements:*

1. Extract the local part of the recipient's email address from the
   |To| header.
2. Use the extracted local part to check if it is present in the
   |Subject| header.
3. The rule should be written in a way that works for any local part 
of

   the email address, not just a specific one.


See the section titled "CAPTURING TAGS USING REGEX NAMED CAPTURE GROUPS" 
in the embedded configuration documentation (perldoc 
Mail::SpamAssassin::Conf) for how to capture a pattern in one rule and 
use it in another. I don't have a working rule for you, but that's the 
mechanism I would use.




--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo@toad.social and many *@billmail.scconsult.com 
addresses)

Not Currently Available For Hire

Extract Local-part from To: Adress to use in spamassassin rule

[Carsten]

Hi @all,

I want to create a SpamAssassin rule that checks if the subject line of 
an email contains the local part of the recipient's email address (the 
part before the @ symbol). For example, if the recipient's email address 
is |i...@example.com|, I want to check if the subject contains the 
phrase "info lorem ipsum". If the recipient's email address is 
|foo...@example.com|, I want to check if the subject contains the phrase 
"foobar lorem ipsum". The rule should be general and adaptable to 
different local parts of email addresses.


*Requirements:*

1. Extract the local part of the recipient's email address from the
   |To| header.
2. Use the extracted local part to check if it is present in the
   |Subject| header.
3. The rule should be written in a way that works for any local part of
   the email address, not just a specific one.

Thank you very much for your suggestions

Re: double backslash in the log messages

[Vincent Lefevre]

On 2024-05-21 13:42:23 -0400, Bill Cole wrote:
> On 2024-05-21 at 11:00:57 UTC-0400 (Tue, 21 May 2024 17:00:57 +0200)
> Vincent Lefevre 
> is rumored to have said:
> 
> > While testing a rule with SpamAssassin 4.0.0 under Debian/stable
> > (I wasn't aware of allow_user_rules yet, but this is not the issue
> > I'm reported):
> > 
> > 2024-05-21T16:42:42.792136+02:00 joooj spamd[219339]: config: not
> > parsing, 'allow_user_rules' is 0: header LOCAL_TO_LORIA ToCc =~
> > /loria\\.fr/i
> > 2024-05-21T16:42:42.793753+02:00 joooj spamd[219339]: config: failed to
> > parse line in /srv/d_joooj/home/vinc17/.spamassassin/user_prefs (line
> > 192): header LOCAL_TO_LORIA ToCc =~ /loria\\.fr/i
> > 
> > while I just had /loria\.fr/i (with a single backslash) in my
> > user_prefs config file.
> > 
> > Is there a reason to have a double backslash in the log messages
> > or is this a bug?
> 
> It is intentional to assure that log messages (which may include strings
> from tainted sources) have all common meta-characters escaped.

After looking at the source (Logger.pm), these are actually control
and non-ASCII characters that are escaped so that they can be printed.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: double backslash in the log messages

[Bill Cole]

On 2024-05-21 at 11:00:57 UTC-0400 (Tue, 21 May 2024 17:00:57 +0200)
Vincent Lefevre 
is rumored to have said:


While testing a rule with SpamAssassin 4.0.0 under Debian/stable
(I wasn't aware of allow_user_rules yet, but this is not the issue
I'm reported):

2024-05-21T16:42:42.792136+02:00 joooj spamd[219339]: config: not 
parsing, 'allow_user_rules' is 0: header LOCAL_TO_LORIA ToCc =~ 
/loria\\.fr/i
2024-05-21T16:42:42.793753+02:00 joooj spamd[219339]: config: failed 
to parse line in /srv/d_joooj/home/vinc17/.spamassassin/user_prefs 
(line 192): header LOCAL_TO_LORIA ToCc =~ /loria\\.fr/i


while I just had /loria\.fr/i (with a single backslash) in my
user_prefs config file.

Is there a reason to have a double backslash in the log messages
or is this a bug?


It is intentional to assure that log messages (which may include strings 
from tainted sources) have all common meta-characters escaped.


--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo@toad.social and many *@billmail.scconsult.com 
addresses)

Not Currently Available For Hire

double backslash in the log messages

[Vincent Lefevre]

While testing a rule with SpamAssassin 4.0.0 under Debian/stable
(I wasn't aware of allow_user_rules yet, but this is not the issue
I'm reported):

2024-05-21T16:42:42.792136+02:00 joooj spamd[219339]: config: not parsing, 
'allow_user_rules' is 0: header LOCAL_TO_LORIA ToCc =~ /loria\\.fr/i
2024-05-21T16:42:42.793753+02:00 joooj spamd[219339]: config: failed to parse 
line in /srv/d_joooj/home/vinc17/.spamassassin/user_prefs (line 192): header 
LOCAL_TO_LORIA ToCc =~ /loria\\.fr/i

while I just had /loria\.fr/i (with a single backslash) in my
user_prefs config file.

Is there a reason to have a double backslash in the log messages
or is this a bug?

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Difference between spamc -L and sa-learn

[Matus UHLAR - fantomas]

On 2024-05-18 at 10:26:54 UTC-0400 (Sat, 18 May 2024 16:26:54 +0200)
Francis Augusto Medeiros-Logeay 
is rumored to have said:

Is there any difference between using spamc -L and sa-learn ?


On 18.05.24 11:41, Bill Cole wrote:
Yes. The compiled-C spamc binary loads no Perl, it just talks over a 
socket to spamd, which is always running and so always has the 
advantage of a warmed-up i/o cache and a permanently loaded set of 
Perl code objects pre-compiled and in RAM; sa-learn has to open and 
compile all of the needed SA Perl code on every launch.



I noticed that the later is way slower.


Yes, it is. It is quite expensive to execute perl and have it load the 
many SpamAssassin modules needed to learn a message.


note that in order for spamc -L to work, spamd must be run with "-l" option 
which allows learning/reporting.


Also, those two may use different databases - sa-learn uses by default 
$HOME/.spamassassin/ (of calling user), spamd depends on how it's run - it 
must run as root and - you need to pass it "-H" parameter without specifying 
directory, to use $HOME/.spamassassin/ of user specified by spamc


Otherwise you need to configure SA to use SQL or LDAP config so they will 
use the same.


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
We are but packets in the Internet of life (userfriendly.org)

[HEADS-UP] Changes to Validity SpamAssassin rules

[Giovanni Bechis]

Hi,
if you are using rules that query Validity rbl (RCVD_IN_VALIDITY_* rules), make 
sure you have updated rules (at least dated 2024-04-23),
otherwise you may encounter in FPs instead of hitting an overlimit response.

  Giovanni


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: Multiple REFUSED logs with sorbs.net ?

[Benny Pedersen]

J Doe skrev den 2024-05-19 23:57:

On 2024-05-17 23:13, Noel Butler wrote:

On 18/05/2024 08:14, J Doe wrote:



Here is an example entry:

10-May-2024 05:34:39.024 lame-servers: info: REFUSED unexpected
RCODE resolving 'rbldns10.sorbs.net/A/IN': 108.59.172.201#53


SORBS has been ultra sensitive like that for a few years now, it 
allows

lookups, then it doesn't, seconds later it does, I suspect an ill
configured DoS protection mechanism that's overly paranoid, but good
luck getting anyone their to listen.


Thank you for your reply ... ok, good to know this is expected 
behaviour

for SORBS.  Like you said, perhaps it is a DoS response ... maybe when
it gets a lot of look ups in a short period of time from an IP it then
throttles subsequent queries ?

- J


also just that to say rbldnsd is not really a dns server

so if that just provide data on this server, it is not dns confirming to 
standards, i lost what part could fails, i just put sorbs in deny on 
spamassassin and live with the missing hits


nothing loosed anyway when its refused imho

Re: Multiple REFUSED logs with sorbs.net ?

[J Doe]

On 2024-05-17 23:13, Noel Butler wrote:


On 18/05/2024 08:14, J Doe wrote:


Hello,

I make use of SpamAssassin 4.0.0 on a low volume e-mail server.  I also
run my own validating resolver with Bind 9.18.27 on the e-mail server.

The only piece of software I have in my e-mail stack that uses SORBS is
SpamAssassin.  I have noticed in my resolver logs multiple entries where
a query of SORBS results in REFUSED results.

Here is an example entry:

10-May-2024 05:34:39.024 lame-servers: info: REFUSED unexpected
RCODE resolving 'rbldns10.sorbs.net/A/IN': 108.59.172.201#53

While some queries succeed and SpamAssassin appears to be able to use
SORBS, there are always *multiple* REFUSED results only for sorbs.net.

Am I exceeding the number of free queries that SORBS allows ?  If so, do
I need to register with SORBS (similar to how SpamHaus requires
registration to use their DQS service) ?  If so, how do I update my SA
configuration ?

Thanks,

- J



SORBS has been ultra sensitive like that for a few years now, it allows
lookups, then it doesn't, seconds later it does, I suspect an ill
configured DoS protection mechanism that's overly paranoid, but good
luck getting anyone their to listen.


Hi Noel,

Thank you for your reply ... ok, good to know this is expected behaviour
for SORBS.  Like you said, perhaps it is a DoS response ... maybe when
it gets a lot of look ups in a short period of time from an IP it then
throttles subsequent queries ?

- J

Re: uridnsbl_skip_domain question

[giovanni]

On 5/17/24 3:17 PM, Matus UHLAR - fantomas wrote:

Hi guys,

I have configured exclusion for some common domains e.g. gov.sk in SA:

uridnsbl_skip_domain [...] gov.sk slovensko.sk

However it seems that that domain is still queried:

  9826  68.951573    127.0.0.1 → 127.0.0.1    DNS 104 Standard query 0xbffe A 
mail.gov.sk.multi.uribl.com OPT

in SA 4 docs I see that:

    uridnsbl_skip_domain domain1 domain2 ...
    Specify a domain, or a number of domains, which should be skipped
    for the URIBL checks.  This is very useful to specify very common
    domains which are not going to be listed in URIBLs.

    In addition to trimmed domain, the full hostname is also checked
    from the list.

Do I have to exclude subdomains for each host too?
(this would kind of defeat the directive imho).

This is SA 3.4.6 (debian 11) which does not have the latter paragraph but I 
assume the difference is only in documentation


From a quick look at the code it seems that subdomains check has been added to 
Mail::SpamAssassin::Plugin::URIDNSBL with commit r1889093 ~10 days after 3.4.6 
release.
In addition to that Mail::SpamAssassin::Plugin::DNSEval honor 
uridnsbl_skip_domain preference only in trunk code.

  Giovanni


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: Difference between spamc -L and sa-learn

[Bill Cole]

On 2024-05-18 at 10:26:54 UTC-0400 (Sat, 18 May 2024 16:26:54 +0200)
Francis Augusto Medeiros-Logeay 
is rumored to have said:


Hi,

Is there any difference between using spamc -L and sa-learn ?


Yes. The compiled-C spamc binary loads no Perl, it just talks over a 
socket to spamd, which is always running and so always has the advantage 
of a warmed-up i/o cache and a permanently loaded set of Perl code 
objects pre-compiled and in RAM; sa-learn has to open and compile all of 
the needed SA Perl code on every launch.



I noticed that the later is way slower.


Yes, it is. It is quite expensive to execute perl and have it load the 
many SpamAssassin modules needed to learn a message.




--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo@toad.social and many *@billmail.scconsult.com 
addresses)

Not Currently Available For Hire

Re: Error parsing sql configuration

[Francis Augusto Medeiros-Logeay]

> On 18 May 2024, at 17:10, Bill Cole  
> wrote:
> 
> On 2024-05-18 at 10:25:28 UTC-0400 (Sat, 18 May 2024 16:25:28 +0200)
> Francis Augusto Medeiros-Logeay 
> is rumored to have said:
> 
>> Hi,
>> 
>> I use Spamassassin 4 on Ubuntu 24.04.
>> 
>> I have configured SQL for storing user preferences. Things work fine, but I 
>> am getting these errors on my logs:
>> 
>> Sat May 18 16:22:21 2024 [75733] info: config: not parsing, administrator 
>> setting: use_pyzor\t1
>> Sat May 18 16:22:21 2024 [75733] info: config: failed to parse line in (sql 
>> config) (line 23): use_pyzor\t1
>> Sat May 18 16:22:21 2024 [75733] info: config: not parsing, administrator 
>> setting: use_razor2\t1
>> Sat May 18 16:22:21 2024 [75733] info: config: failed to parse line in (sql 
>> config) (line 28): use_razor2\t1
>> 
>> My query is pretty standard:
>> 
>> user_scores_sql_custom_query SELECT preference,value FROM 
>> spam_assassin_userpref WHERE username = _USERNAME_ OR username = '$GLOBAL' 
>> OR username = CONCAT('%',_DOMAIN_) ORDER BY username ASC
>> 
>> Is there a bug when parsing the preferences from sql?
> 
> It's not really a parsing error, it's a configuration error. You cannot set 
> "use_pyzor" or "use_razor" in user preferences, as they are both restricted 
> to system-wide config.


Thank you Bill. I kinda suspected that, but I am using a plugin called 
«sauserpref» on roundcube that actually set these on the database, so I was a 
bit unsure.

Best,
Francis 

Re: Error parsing sql configuration

[Bill Cole]

On 2024-05-18 at 10:25:28 UTC-0400 (Sat, 18 May 2024 16:25:28 +0200)
Francis Augusto Medeiros-Logeay 
is rumored to have said:


Hi,

I use Spamassassin 4 on Ubuntu 24.04.

I have configured SQL for storing user preferences. Things work fine, 
but I am getting these errors on my logs:


Sat May 18 16:22:21 2024 [75733] info: config: not parsing, 
administrator setting: use_pyzor\t1
Sat May 18 16:22:21 2024 [75733] info: config: failed to parse line in 
(sql config) (line 23): use_pyzor\t1
Sat May 18 16:22:21 2024 [75733] info: config: not parsing, 
administrator setting: use_razor2\t1
Sat May 18 16:22:21 2024 [75733] info: config: failed to parse line in 
(sql config) (line 28): use_razor2\t1


My query is pretty standard:

user_scores_sql_custom_query SELECT preference,value FROM 
spam_assassin_userpref WHERE username = _USERNAME_ OR username = 
'$GLOBAL' OR username = CONCAT('%',_DOMAIN_) ORDER BY username ASC


Is there a bug when parsing the preferences from sql?


It's not really a parsing error, it's a configuration error. You cannot 
set "use_pyzor" or "use_razor" in user preferences, as they are both 
restricted to system-wide config.





--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo@toad.social and many *@billmail.scconsult.com 
addresses)

Not Currently Available For Hire

Difference between spamc -L and sa-learn

[Francis Augusto Medeiros-Logeay]

Hi,

Is there any difference between using spamc -L and sa-learn ? I noticed that 
the later is way slower. I don’t use a journal for local updating, so both 
write directly to the database.

Best,

Francis 

Error parsing sql configuration

[Francis Augusto Medeiros-Logeay]

Hi,

I use Spamassassin 4 on Ubuntu 24.04.

I have configured SQL for storing user preferences. Things work fine, but I am 
getting these errors on my logs:

Sat May 18 16:22:21 2024 [75733] info: config: not parsing, administrator 
setting: use_pyzor\t1
Sat May 18 16:22:21 2024 [75733] info: config: failed to parse line in (sql 
config) (line 23): use_pyzor\t1
Sat May 18 16:22:21 2024 [75733] info: config: not parsing, administrator 
setting: use_razor2\t1
Sat May 18 16:22:21 2024 [75733] info: config: failed to parse line in (sql 
config) (line 28): use_razor2\t1

My query is pretty standard: 

user_scores_sql_custom_query SELECT preference,value FROM 
spam_assassin_userpref WHERE username = _USERNAME_ OR username = '$GLOBAL' OR 
username = CONCAT('%',_DOMAIN_) ORDER BY username ASC

Is there a bug when parsing the preferences from sql? 

Best,
Francis 

Re: Multiple REFUSED logs with sorbs.net ?

[Noel Butler]

On 18/05/2024 08:14, J Doe wrote:


Hello,

I make use of SpamAssassin 4.0.0 on a low volume e-mail server.  I also
run my own validating resolver with Bind 9.18.27 on the e-mail server.

The only piece of software I have in my e-mail stack that uses SORBS is
SpamAssassin.  I have noticed in my resolver logs multiple entries 
where

a query of SORBS results in REFUSED results.

Here is an example entry:

10-May-2024 05:34:39.024 lame-servers: info: REFUSED unexpected
RCODE resolving 'rbldns10.sorbs.net/A/IN': 108.59.172.201#53

While some queries succeed and SpamAssassin appears to be able to use
SORBS, there are always *multiple* REFUSED results only for sorbs.net.

Am I exceeding the number of free queries that SORBS allows ?  If so, 
do

I need to register with SORBS (similar to how SpamHaus requires
registration to use their DQS service) ?  If so, how do I update my SA
configuration ?

Thanks,

- J


SORBS has been ultra sensitive like that for a few years now, it allows 
lookups, then it doesn't, seconds later it does, I suspect an ill 
configured DoS protection mechanism that's overly paranoid, but good 
luck getting anyone their to listen.


--
Regards,
Noel Butler

Multiple REFUSED logs with sorbs.net ?

[J Doe]

Hello,

I make use of SpamAssassin 4.0.0 on a low volume e-mail server.  I also
run my own validating resolver with Bind 9.18.27 on the e-mail server.

The only piece of software I have in my e-mail stack that uses SORBS is
SpamAssassin.  I have noticed in my resolver logs multiple entries where
a query of SORBS results in REFUSED results.

Here is an example entry:

10-May-2024 05:34:39.024 lame-servers: info: REFUSED unexpected
RCODE resolving 'rbldns10.sorbs.net/A/IN': 108.59.172.201#53

While some queries succeed and SpamAssassin appears to be able to use
SORBS, there are always *multiple* REFUSED results only for sorbs.net.

Am I exceeding the number of free queries that SORBS allows ?  If so, do
I need to register with SORBS (similar to how SpamHaus requires
registration to use their DQS service) ?  If so, how do I update my SA
configuration ?

Thanks,

- J

uridnsbl_skip_domain question

[Matus UHLAR - fantomas]

Hi guys,

I have configured exclusion for some common domains e.g. gov.sk in SA:

uridnsbl_skip_domain [...] gov.sk slovensko.sk

However it seems that that domain is still queried:

 9826  68.951573127.0.0.1 → 127.0.0.1DNS 104 Standard query 0xbffe A 
mail.gov.sk.multi.uribl.com OPT

in SA 4 docs I see that:

   uridnsbl_skip_domain domain1 domain2 ...
   Specify a domain, or a number of domains, which should be skipped
   for the URIBL checks.  This is very useful to specify very common
   domains which are not going to be listed in URIBLs.

   In addition to trimmed domain, the full hostname is also checked
   from the list.

Do I have to exclude subdomains for each host too?
(this would kind of defeat the directive imho).

This is SA 3.4.6 (debian 11) which does not have the latter paragraph but I 
assume the difference is only in documentation


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
You have the right to remain silent. Anything you say will be misquoted,
then used against you.

Unsubscribe

[Anshul Chauhan]

Re: SA treats percentage spaces wording as uri

[Bill Cole]

On 2024-05-13 at 20:09:33 UTC-0400 (Tue, 14 May 2024 10:09:33 +1000)
Noel Butler 
is rumored to have said:

This morning one of our ent_domains DMARC weekly report from a third 
party was listed as spam by SA which took the wording  
Not_percent-twenty_Resolved and passed it off to URI checks adding 
dot.com to it when there is no dot com after it, and a raw message 
search of that message in less in console confirms it.


Context is important. If SA is mis-parsing a message, we really need to 
see the message to understand why. There's nothing obviously magic about 
that string.



Problem with the code that scans the content for things like URI's?


Likely.

That code is intentionally loose. It is intended to turn anything that 
any MUA might consider a clickable link into the same functional URI 
that a MUA would. This creates a fundamental tension between 
completeness and correctness. SA leans towards completeness but if it is 
doing something harmful we'd like to fix that. It would be particularly 
important to fix it if the result was a hit on a substantial rule, but 
it is not as important to avoid checking bogus URIs that will never hit 
anything anyway.



It shouldn't be assuming there's a TLD after it.


I agree. That's a step too far. The days when appending .com was a 
reasonable tactic for qualifying hostnames are long gone.


--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo@toad.social and many *@billmail.scconsult.com 
addresses)

Not Currently Available For Hire

Re: SA treats percentage spaces wording as uri

[Shawn Iverson]

On Mon, May 13, 2024 at 8:10 PM Noel Butler  wrote:

> This morning one of our ent_domains DMARC weekly report from a third party
> was listed as spam by SA which took the wording
>  Not_percent-twenty_Resolved and passed it off to URI checks adding
> dot.com to it when there is no dot com after it, and a raw message search
> of that message in less in console confirms it.
>
If you place this badly formatted string as a link, URI checks appear to
take it and append a dot com to it as well.  I'd imagine that the word
itself isn't important and can be arbitrary.

http:undefined

Re: SA treats percentage spaces wording as uri

[Matus UHLAR - fantomas]

On 14.05.24 10:09, Noel Butler wrote:
This morning one of our ent_domains DMARC weekly report from a third 
party was listed as spam by SA which took the wording  
Not_percent-twenty_Resolved and passed it off to URI checks adding 
dot.com to it when there is no dot com after it, and a raw message 
search of that message in less in console confirms it.


Problem with the code that scans the content for things like URI's? It 
shouldn't be assuming there's a TLD after it.


are you sure that .com was not in the original mail?
Some MUAs like to change everything possible to an URL even if you don't see 
it.


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
He who laughs last thinks slowest.

SA treats percentage spaces wording as uri

[Noel Butler]

This morning one of our ent_domains DMARC weekly report from a third 
party was listed as spam by SA which took the wording  
Not_percent-twenty_Resolved and passed it off to URI checks adding 
dot.com to it when there is no dot com after it, and a raw message 
search of that message in less in console confirms it.


Problem with the code that scans the content for things like URI's? It 
shouldn't be assuming there's a TLD after it.


--
Regards,
Noel Butler

Re: dkim https://16years.secvuln.info/

[Bill Cole]

On 2024-05-13 at 08:09:04 UTC-0400 (Mon, 13 May 2024 14:09:04 +0200)
Benny Pedersen 
is rumored to have said:

i write here so in hope to start a debate on it, is there a code 
change any where to handle this ?


That's not a SA issue. Nothing SA does can fix it

The change (in Debian) that fixed that vulnerability was released 16 
years ago. It is up to sysadmins to pay attention and deploy fixes when 
they are available.  If people are still using bad keys generated 16 
years ago, they are failing to do that. We can't fix it.


The problem being cited in 2024 is 16 years of incompetent system 
administration, not bad code or distribution config.



--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo@toad.social and many *@billmail.scconsult.com 
addresses)

Not Currently Available For Hire

dkim https://16years.secvuln.info/

[Benny Pedersen]

i write here so in hope to start a debate on it, is there a code change 
any where to handle this ?

Re: Score 0.001

[Thomas Barth via users]

Am 2024-05-13 04:33, schrieb jdow:

Um, "FORGED_SPF_HELO"? Are you sure this message is from MS?

{^_^}


The mail/report is authentic. They already corrected this "error" or 
changed the sending server. In today's report FORGED_SPF_HELO is 0.001 
and the score is below 5 :)



On 20240512 06:56:59, Thomas Barth wrote:


Am 2024-05-12 12:39, schrieb Greg Troxel:


I would suggest that if Debian is modifying the default config
from 5 to
6.31, then probably they should not be doing that.


This is a status of dmarc-report from microsoft today

X-Spam-Status: Yes, score=5.938 tagged_above=2 required=6.31
tests=[ARC_SIGNED=0.001, ARC_VALID=0.001,
BASE64_LENGTH_78_79=0.1,
BASE64_LENGTH_79_INF=2.019, DKIMWL_WL_MED=-0.001,
DKIM_SIGNED=0.1,
DKIM_VALID=-0.1, DMARC_PASS=-0.001, FORGED_SPF_HELO=1,
HTML_MESSAGE=0.001,
MIME_BASE64_TEXT=0.001, MIME_HTML_MOSTLY=0.1,
MPART_ALT_DIFF=0.724,
PYZOR_CHECK=1.985, RCVD_IN_MSPIKE_H2=-0.001,
SPF_HELO_PASS=-0.001,
T_TVD_MIME_NO_HEADERS=0.01]

A strike level of 5 is too low for microsoft mails ;-)

Re: Score 0.001

[jdow]

Um, "FORGED_SPF_HELO"? Are you sure this message is from MS?

{^_^}

On 20240512 06:56:59, Thomas Barth wrote:

Am 2024-05-12 12:39, schrieb Greg Troxel:

I would suggest that if Debian is modifying the default config from 5 to
6.31, then probably they should not be doing that.


This is a status of dmarc-report from microsoft today

X-Spam-Status: Yes, score=5.938 tagged_above=2 required=6.31
    tests=[ARC_SIGNED=0.001, ARC_VALID=0.001, BASE64_LENGTH_78_79=0.1,
    BASE64_LENGTH_79_INF=2.019, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1,
    DKIM_VALID=-0.1, DMARC_PASS=-0.001, FORGED_SPF_HELO=1, HTML_MESSAGE=0.001,
    MIME_BASE64_TEXT=0.001, MIME_HTML_MOSTLY=0.1, MPART_ALT_DIFF=0.724,
    PYZOR_CHECK=1.985, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001,
    T_TVD_MIME_NO_HEADERS=0.01]

A strike level of 5 is too low for microsoft mails ;-)

Re: Score 0.001

[Benny Pedersen]

Thomas Barth skrev den 2024-05-12 15:56:

Am 2024-05-12 12:39, schrieb Greg Troxel:
I would suggest that if Debian is modifying the default config from 5 
to

6.31, then probably they should not be doing that.


This is a status of dmarc-report from microsoft today

X-Spam-Status: Yes, score=5.938 tagged_above=2 required=6.31
tests=[ARC_SIGNED=0.001, ARC_VALID=0.001, BASE64_LENGTH_78_79=0.1,
BASE64_LENGTH_79_INF=2.019, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1,
DKIM_VALID=-0.1, DMARC_PASS=-0.001, FORGED_SPF_HELO=1, 
HTML_MESSAGE=0.001,

MIME_BASE64_TEXT=0.001, MIME_HTML_MOSTLY=0.1, MPART_ALT_DIFF=0.724,
PYZOR_CHECK=1.985, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001,
T_TVD_MIME_NO_HEADERS=0.01]

A strike level of 5 is too low for microsoft mails ;-)


X-Spam-Status: No, score=-0.719 tagged_above=-999 required=5
tests=[AUTHRES_DKIM_PASS=-0.5, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.001,
KAM_NUMSUBJECT=0.4, MAILING_LIST_MULTI=-0.1, RCVD_IN_MSPIKE_H4=-0.01,
RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.1, SPF_PASS=-0.1,
USER_IN_DEF_SPF_WL=-0.2] autolearn=unavailable autolearn_force=no

AuthRes is nice :)

Re: Score 0.001

[Thomas Barth]

Am 2024-05-12 12:39, schrieb Greg Troxel:
I would suggest that if Debian is modifying the default config from 5 
to

6.31, then probably they should not be doing that.


This is a status of dmarc-report from microsoft today

X-Spam-Status: Yes, score=5.938 tagged_above=2 required=6.31
tests=[ARC_SIGNED=0.001, ARC_VALID=0.001, BASE64_LENGTH_78_79=0.1,
BASE64_LENGTH_79_INF=2.019, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1,
DKIM_VALID=-0.1, DMARC_PASS=-0.001, FORGED_SPF_HELO=1, 
HTML_MESSAGE=0.001,

MIME_BASE64_TEXT=0.001, MIME_HTML_MOSTLY=0.1, MPART_ALT_DIFF=0.724,
PYZOR_CHECK=1.985, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001,
T_TVD_MIME_NO_HEADERS=0.01]

A strike level of 5 is too low for microsoft mails ;-)

Re: Score 0.001

[Matus UHLAR - fantomas]

On 12.05.24 06:39, Greg Troxel wrote:

I would suggest that if Debian is modifying the default config from 5 to
6.31, then


as it was already said, it's not Debian, it's default score in amavis.
Even the original header is in the amavis format:


X-Spam-Status: No, score=3.999 tagged_above=2 required=6.31
tests=[DMARC_MISSING=0.001, FSL_BULK_SIG=0.001, 


Amavis has some more scores than stock SA, of course they can be modified if 
your scanner is well trained.




--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Linux - It's now safe to turn on your computer.
Linux - Teraz mozete pocitac bez obav zapnut.

Re: Score 0.001

[Greg Troxel]

I would suggest that if Debian is modifying the default config from 5 to
6.31, then

  probably they should not be doing that.  as a packager, I fix bugs
  (and file upstream bug reports), but it's usually linuxy
  nonportability things that are clearly bugs (test ==, hardcoded lists
  of accepted operating systems, etc.).  This is a difference in
  judgement.

  if they are applying a difference in judgement, the package
  description should disclose this really clearly.  Hard to tell what's
  going on, but this appears to be new to most people here.
  

Re: Score 0.001

[Thomas Barth]

Am 2024-05-12 01:08, schrieb jdow:

Methinks this is a perfect example of "one man's spam is another man's
ham." Or in my case, "A woman's spam is often a man's ham."


I like spam when it's well designed. That's why I no longer reject it on 
my newly set up mail server. I just want them all to be saved in the 
junk folder. I sometimes admire the creativity of spammers to attract 
attention to a ridiculous product :)

Re: Whitelist rules should never pass on SPF fail

[Noel Butler]

On 11/05/2024 03:40, Bill Cole wrote:

So what? domain owners state hard fail it SHOULD be hard failed, 
irrespective of if YOU think you know better than THEM or not, if we 
hardfail we accept the risks that come with it.


In practice, there is a prioritizing of whose wishes I prioritize on 
the receiving systems I work with. If my customer wants to receive the 
mail and the individual generating the mail is not generating that 
desire fraudulently, I don't care much about what the domain owner 
says.


I hope you have an indemnity clause in your contracts (or written 
statement from them) to legally protect you, and your professional 
indemnity insurance (or your countries version of it) is current...


I do not work for the domain owners of the world and I am not obligated 
to enforce their usage rules on their users.


Obligated no, its your network, your rules, but honouring them is the 
correct "good netizen" thing to do.


I'm sure the crime gangs and spammers reading this list greatly 
appreciate you telling them they got better chances with you then most 
:P


Obviously I take their input seriously when trying to detect fraud but 
I've seen too many cases of "-all" being used with incomplete or 
obsolete lists of "permitted" hosts to accept that they know all of the 
places their mail gets generated.


The idea of using -all is not just configuring it and forgetting it, 
it's part of the accepted risk that if you change something, you change 
your SPF statements too, if they forget, the complaints of blocked mail 
should prompt them to fix it, or if they are just flat out too damn 
lazy, then they get what they deserve.


Adherence has improved out of sight in past 5 to 10 years, and I've seen 
no problems caused by SPF, I can't remember the last time we had one.


I've also given up all hope of getting the few places that are still 
doing transparent forwarding to adopt SRS or any other mechanisms to 
avoid SPF breakage to ever change.


I guess the traffic with them is low, if it was high, blocking would 
likely get them off their buts.


--
Regards,
Noel Butler

Re: Score 0.001

[jdow]

On 20240511 14:56:51, Greg Troxel wrote:

Thomas Barth  writes:


Am 2024-05-11 21:54, schrieb Bill Cole:

I have no idea who the Debian "spam analysts" are but I am certain
that they are not doing any sort of data-driven dynamic adjustments
of scores based on a threshold of 6.3 nor are they (obviously)
adjusting that threshold daily based on current scores.

I found the passage in my old Postfix book. The author writes: "It is
recommended not to carelessly set the value of $sa_kill_level_deflt to
any fantasy values. The score of 6.31 is not arbitrarily chosen, but
the statistically calculated optimum for the best possible spam filter
rate with as few false positives as possible. If you increase the
value, more spam will get through; if you lower it, your false
positives will increase."

The comments about adjustments are true, but the idea that it is optimum
is flat-out nonsensical.

The key question is how you weight a false positive compared to a false
negative.  Only after you decided that can you pick an optimium, for a
given corpus of already-received mail.


It may be that the value is outdated, but that is for the maintainers
of the relevant Debian package to decide. I'll just adapt my rules to
this one value.

That is an odd position.  It is very easy to set the threshold in a
local config.   Deciding instead to adjust scores to an oddball
threshold seems bizarre to me.

Personally, I don't use the 5, but instead have shades of grey, where

=1 is binned into mailboxes that are "maybe spam" through "very likely"

spam, and at some score, I reject at the MTA level.

I find that legit mail shows up in e.g. spam.2 (>= 2 and < 3), but it is
almost never mail that I would be upset to have missed (but I don't) or
mail that I would be upset to not get in a timely manner (I only see it
every day or so).  However, this really drops the FN rate of spam in my
INBOX, which matters a lot to me.Basically I consider a FP into my
"spam.1" mailbox, as long as it isn't really important to me, to be not
a big deal at all, and I'd rather have 10 or those than 1 FN in my
INBOX.  But, actually MTA-rejecting mail that I shouldn't, a FP at that
level, is a big deal, and I avoid it.  I think it's about one message a
year -- and while it's ham, it's very spammy ham.


Methinks this is a perfect example of "one man's spam is another man's ham." Or 
in my case, "A woman's spam is often a man's ham."


{^_-}   <- hopelessly square and antiquarian by today's standards. e.g. the 
XXXlist wars.

Re: Score 0.001

[Thomas Barth]

Am 2024-05-11 23:49, schrieb Vincent Lefevre:

The value 6.31 does not even appear in the spamassassin source
package.


Sorry, the values are overwritten via the Amavis defaults.

cat /etc/debian_version
10.13
egrep -nri "sa_tag_level_deflt|sa_kill_level_deflt" /etc
/etc/amavis/conf.d/20-debian_defaults:36:$sa_tag_level_deflt  = 2.0;  # 
add spam info headers if at, or above that level
/etc/amavis/conf.d/20-debian_defaults:38:$sa_kill_level_deflt = 6.31; # 
triggers spam evasive actions


cat /etc/debian_version
12.5
egrep -nri "sa_tag_level_deflt|sa_kill_level_deflt" /etc
/etc/amavis/conf.d/20-debian_defaults:36:$sa_tag_level_deflt  = 2.0;  # 
add spam info headers if at, or above that level
/etc/amavis/conf.d/20-debian_defaults:38:$sa_kill_level_deflt = 6.31; # 
triggers spam evasive actions

Re: Score 0.001

[Greg Troxel]

Thomas Barth  writes:

> Am 2024-05-11 21:54, schrieb Bill Cole:
>> I have no idea who the Debian "spam analysts" are but I am certain
>> that they are not doing any sort of data-driven dynamic adjustments
>> of scores based on a threshold of 6.3 nor are they (obviously)
>> adjusting that threshold daily based on current scores.
>
> I found the passage in my old Postfix book. The author writes: "It is
> recommended not to carelessly set the value of $sa_kill_level_deflt to
> any fantasy values. The score of 6.31 is not arbitrarily chosen, but
> the statistically calculated optimum for the best possible spam filter
> rate with as few false positives as possible. If you increase the
> value, more spam will get through; if you lower it, your false
> positives will increase."

The comments about adjustments are true, but the idea that it is optimum
is flat-out nonsensical.

The key question is how you weight a false positive compared to a false
negative.  Only after you decided that can you pick an optimium, for a
given corpus of already-received mail.

> It may be that the value is outdated, but that is for the maintainers
> of the relevant Debian package to decide. I'll just adapt my rules to
> this one value.

That is an odd position.  It is very easy to set the threshold in a
local config.   Deciding instead to adjust scores to an oddball
threshold seems bizarre to me.

Personally, I don't use the 5, but instead have shades of grey, where
>=1 is binned into mailboxes that are "maybe spam" through "very likely"
spam, and at some score, I reject at the MTA level.

I find that legit mail shows up in e.g. spam.2 (>= 2 and < 3), but it is
almost never mail that I would be upset to have missed (but I don't) or
mail that I would be upset to not get in a timely manner (I only see it
every day or so).  However, this really drops the FN rate of spam in my
INBOX, which matters a lot to me.Basically I consider a FP into my
"spam.1" mailbox, as long as it isn't really important to me, to be not
a big deal at all, and I'd rather have 10 or those than 1 FN in my
INBOX.  But, actually MTA-rejecting mail that I shouldn't, a FP at that
level, is a big deal, and I avoid it.  I think it's about one message a
year -- and while it's ham, it's very spammy ham.

Re: Score 0.001

[Vincent Lefevre]

On 2024-05-11 20:26:59 +0200, Thomas Barth wrote:
> Am 2024-05-11 19:24, schrieb Loren Wilton:
[...]
> > > found in
> > > 
> > > X-Spam-Status: No, score=5.908 tagged_above=2 required=6.31
> > > tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
> > > DKIM_VALID_EF=-0.1, DMARC_PASS=-0.001, FSL_BULK_SIG=0.001,
> > > HTML_MESSAGE=0.001, RAZOR2_CF_RANGE_51_100=2.43,
> > > RAZOR2_CHECK=1.729,
> > > SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_ABUSE_SURBL=1.948]
> > 
> > Why is your score threshold for spam 6.31? By default it is 5, and that
> > message would have been spam.
> 
> 6.31 has been the default value on a Debian system for ages and is based on
> the experience of the “spam analysts”.

No, Debian has not changed the default, which is 5.0, as I can see
for your message (here, on a Debian/stable machine):

X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on joooj.vinc17.net
X-Spam-Status: No, score=-17.4 required=5.0 tests=BAYES_00,DKIM_SIGNED,
DKIM_VALID,DKIM_VALID_AU,DMARC_PASS,HEADER_FROM_DIFFERENT_DOMAINS,
MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,RCVD_IN_MSPIKE_H4,
RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,SPF_PASS,USER_IN_DEF_SPF_WL
autolearn=ham autolearn_force=no version=4.0.0

The value 6.31 does not even appear in the spamassassin source
package.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Score 0.001

[Thomas Barth]

Am 2024-05-11 21:54, schrieb Bill Cole:
I have no idea who the Debian "spam analysts" are but I am certain that 
they are not doing any sort of data-driven dynamic adjustments of 
scores based on a threshold of 6.3 nor are they (obviously) adjusting 
that threshold daily based on current scores.


I found the passage in my old Postfix book. The author writes: "It is 
recommended not to carelessly set the value of $sa_kill_level_deflt to 
any fantasy values. The score of 6.31 is not arbitrarily chosen, but the 
statistically calculated optimum for the best possible spam filter rate 
with as few false positives as possible. If you increase the value, more 
spam will get through; if you lower it, your false positives will 
increase."
It may be that the value is outdated, but that is for the maintainers of 
the relevant Debian package to decide. I'll just adapt my rules to this 
one value.

Re: Score 0.001

[Bill Cole]

On 2024-05-11 at 14:26:59 UTC-0400 (Sat, 11 May 2024 20:26:59 +0200)
Thomas Barth 
is rumored to have said:


Hello

Am 2024-05-11 19:24, schrieb Loren Wilton:

Can I just take the names of the rules?

e.g. at least two checks should fire:

meta MULTIPLE_TESTS (( RAZOR2_CF_RANGE_51_100 + RAZOR2_CHECK + 
URIBL_ABUSE_SURBL) > 1)

score MULTIPLE_TESTS 1

found in

X-Spam-Status: No, score=5.908 tagged_above=2 required=6.31
tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
DKIM_VALID_EF=-0.1, DMARC_PASS=-0.001, FSL_BULK_SIG=0.001,
HTML_MESSAGE=0.001, RAZOR2_CF_RANGE_51_100=2.43, 
RAZOR2_CHECK=1.729,

SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_ABUSE_SURBL=1.948]


Why is your score threshold for spam 6.31? By default it is 5, and 
that message would have been spam.


6.31 has been the default value on a Debian system for ages and is 
based on the experience of the “spam analysts”. That's how I 
remember it. I have therefore retained this value. Who introduced the 
default value of 5? Spamassassin itself, because spam is getting 
better and better and fewer rules apply?


5.0 has been the default threshold in the distribution forever and that 
value is an assumption in the dynamic scoring and RuleQA service which 
adjusts scores to their optimal values daily based on the latest results 
submitted by masscheck contributors.


I have no idea who the Debian "spam analysts" are but I am certain that 
they are not doing any sort of data-driven dynamic adjustments of scores 
based on a threshold of 6.3 nor are they (obviously) adjusting that 
threshold daily based on current scores. The only reason I can see for 
boosting the threshold is if there is an additional set of rules being 
used with a significant number of the non-standard low-S/O rules. For 
example, if you use KAM rules (which are not part of the RuleQA process) 
you will have a lot of rule hits on legit mail and you can either boost 
the threshold or do a lot of local-specific FP mitigation.


On systems I manage I mostly use a *lower* threshold, because I apply 
more active site-specific rule management (and FP avoidance) than most 
systems ever receive.




--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo@toad.social and many *@billmail.scconsult.com 
addresses)

Not Currently Available For Hire

Re: Score 0.001

[Thomas Barth]

Hello

Am 2024-05-11 19:24, schrieb Loren Wilton:

Can I just take the names of the rules?

e.g. at least two checks should fire:

meta MULTIPLE_TESTS (( RAZOR2_CF_RANGE_51_100 + RAZOR2_CHECK + 
URIBL_ABUSE_SURBL) > 1)

score MULTIPLE_TESTS 1

found in

X-Spam-Status: No, score=5.908 tagged_above=2 required=6.31
tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
DKIM_VALID_EF=-0.1, DMARC_PASS=-0.001, FSL_BULK_SIG=0.001,
HTML_MESSAGE=0.001, RAZOR2_CF_RANGE_51_100=2.43, 
RAZOR2_CHECK=1.729,

SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_ABUSE_SURBL=1.948]


Why is your score threshold for spam 6.31? By default it is 5, and that 
message would have been spam.


6.31 has been the default value on a Debian system for ages and is based 
on the experience of the “spam analysts”. That's how I remember it. I 
have therefore retained this value. Who introduced the default value of 
5? Spamassassin itself, because spam is getting better and better and 
fewer rules apply?




The meta you suggest would have fired and added a point, but only 
because the combined score for the rules it mentions added up to > 1.0. 
Since every single one has a score > 1.0, the rule would have fired if 
any single one or any combination of those rules fired. Is that what 
you intended? It would not have fired if you had picked SPF_HELO_NONE, 
SPF_PASS, and FSL_BULK_SIG, as that only adds up to 0.003.


Most commonly a meta for "multiple rules fired" would have used the && 
operator, for instance:


meta MY_MANY_RULESSPF_HELO_NONE && SPF_PASS && FSL_BULK_SIG
describe   MY_MANY_RULESSeveral random rules hit
scoreMY_MANY_RULES1


Your metarule says: all three subrules must match. My rule says, that at 
least 2 subrules must match. Here it can be A+B+C, A+B, A+C, B+C. In my 
rule the matches (trues) are counted. I have taken this rule from the 
wiki. Please look here at meta rules 
https://cwiki.apache.org/confluence/display/SPAMASSASSIN/writingrules

Re: Score 0.001

[Loren Wilton]

Can I just take the names of the rules?

e.g. at least two checks should fire:

meta MULTIPLE_TESTS (( RAZOR2_CF_RANGE_51_100 + RAZOR2_CHECK + 
URIBL_ABUSE_SURBL) > 1)

score MULTIPLE_TESTS 1

found in

X-Spam-Status: No, score=5.908 tagged_above=2 required=6.31
tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
DKIM_VALID_EF=-0.1, DMARC_PASS=-0.001, FSL_BULK_SIG=0.001,
HTML_MESSAGE=0.001, RAZOR2_CF_RANGE_51_100=2.43, RAZOR2_CHECK=1.729,
SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_ABUSE_SURBL=1.948]


Why is your score threshold for spam 6.31? By default it is 5, and that 
message would have been spam.


The meta you suggest would have fired and added a point, but only because 
the combined score for the rules it mentions added up to > 1.0. Since every 
single one has a score > 1.0, the rule would have fired if any single one or 
any combination of those rules fired. Is that what you intended? It would 
not have fired if you had picked SPF_HELO_NONE, SPF_PASS, and FSL_BULK_SIG, 
as that only adds up to 0.003.


Most commonly a meta for "multiple rules fired" would have used the && 
operator, for instance:


meta MY_MANY_RULESSPF_HELO_NONE && SPF_PASS && FSL_BULK_SIG
describe   MY_MANY_RULESSeveral random rules hit
scoreMY_MANY_RULES1

Re: Score 0.001

[Thomas Barth]

Hi guys,

thank you all for your advice!

Am 2024-05-10 22:39, schrieb Bowie Bailey:
The rules with the low scores are not intended to contribute to the 
spam score for the email.  They only have a defined score at all 
because if the score is 0, SA will not run the rule.


It works like this:

Rule A has a score of 0.001
Rule B has a score of 0.001
Rule C is a meta that matches if both A and B match, and has a score of 
5


It doesn't matter how small the scores are for rule A and B.  The only 
thing that matters is the score for rule C.  If only A matches, then it 
adds 0.001 to the score and the email is not spam.  If only B matches, 
then you get the same result.  But if they both match, then you get a 
score of 5.002.  The entire point of the 0.001 score is that you could 
match 100 of these rules and not affect the spam score.


These rules are generally things like, "the email has HTML", "there is 
an SPF check", "there is a google drive link", etc.  On their own, they 
do not mean anything, but metas can combine these low-scored rules into 
meaningful patterns that are then given larger scores.



Can I just take the names of the rules?

e.g. at least two checks should fire:

meta MULTIPLE_TESTS (( RAZOR2_CF_RANGE_51_100 + RAZOR2_CHECK + 
URIBL_ABUSE_SURBL) > 1)

score MULTIPLE_TESTS 1

found in

X-Spam-Status: No, score=5.908 tagged_above=2 required=6.31
tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
DKIM_VALID_EF=-0.1, DMARC_PASS=-0.001, FSL_BULK_SIG=0.001,
HTML_MESSAGE=0.001, RAZOR2_CF_RANGE_51_100=2.43, RAZOR2_CHECK=1.729,
SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_ABUSE_SURBL=1.948]

Re: Score 0.001

[Bowie Bailey]

On 5/10/2024 2:57 AM, Thomas Barth wrote:

Am 2024-05-10 06:19, schrieb Reindl Harald (privat):

Am 10.05.24 um 00:05 schrieb Thomas Barth:

Am 2024-05-09 21:41, schrieb Loren Wilton:
Low-score tests are neither spam nor ham signs by themselves. They 
can be used in metas in conjunction with other indicators to help 
determine ham or spam. A zero value indicates that a rule didn't 
hit and the sign is not present. A small score indicates that the 
rule did hit, so the sign it is detecting is present.


0.001 seems to be the default lowest value. Is it possible to change 
it to 0.01 or 0.1?


what do you not understand in meta tests?
it's irrelevant if it's 0.001 or 0.01

these rules are used in combination with other rules

HTML_MESSAGE or SPF_HELO_NONE alone don't mean anything and so it 
must not score higher - it makes only sense combined with other rules


Most of the messages I receive only have a few hits because they 
hardly differ from a regular e-mail. That's why I want to assign a 
higher value to the individual tests. I don't know how many of the 
possible tests with a value of only 0.001 exist. With this value, 
theoretically 1000 different tests would have to be positive in order 
to achieve a total value of 1. Therefore, it is not irrelevant whether 
I have a minimum value of 0.001 or 0.1. I would even go further and 
say if there are more than 10 tests with a positive value: Spam! 
Either the strike level is reached or there are more than 10 tests 
with a positive value. So now I repeat my question: is it possible to 
increase the minimum value to 0.1 by default?


Going through this thread, I note that a few people have said "they are 
used in metas", but no one has actually given an example of how that works.


The rules with the low scores are not intended to contribute to the spam 
score for the email.  They only have a defined score at all because if 
the score is 0, SA will not run the rule.


It works like this:

Rule A has a score of 0.001
Rule B has a score of 0.001
Rule C is a meta that matches if both A and B match, and has a score of 5

It doesn't matter how small the scores are for rule A and B.  The only 
thing that matters is the score for rule C.  If only A matches, then it 
adds 0.001 to the score and the email is not spam.  If only B matches, 
then you get the same result.  But if they both match, then you get a 
score of 5.002.  The entire point of the 0.001 score is that you could 
match 100 of these rules and not affect the spam score.


These rules are generally things like, "the email has HTML", "there is 
an SPF check", "there is a google drive link", etc.  On their own, they 
do not mean anything, but metas can combine these low-scored rules into 
meaningful patterns that are then given larger scores.


--
Bowie

Re: Score 0.001

[Bill Cole]

On 2024-05-10 at 14:15:56 UTC-0400 (Fri, 10 May 2024 14:15:56 -0400)
Bill Cole 
is rumored to have said:

> On 2024-05-09 at 18:19:14 UTC-0400 (Thu, 9 May 2024 15:19:14 -0700)
> jdow 
> is rumored to have said:
>
>> On 20240509 15:05:46, Thomas Barth wrote:
>>> Am 2024-05-09 21:41, schrieb Loren Wilton:
 Low-score tests are neither spam nor ham signs by themselves. They can be 
 used in metas in conjunction with other indicators to help determine ham 
 or spam. A zero value indicates that a rule didn't hit and the sign is not 
 present. A small score indicates that the rule did hit, so the sign it is 
 detecting is present.
>>>
>>> 0.001 seems to be the default lowest value. Is it possible to change it to 
>>> 0.01 or 0.1?
>
> Sure. It's just a number.

Clarifying; You can change any score yourself on your own system locally if you 
like, but to make no rule ever score 0.001 you'd need to fix the scores for all 
low-score rules every time that you run sa-update. As John Hardin says, we will 
not be changing the default to 0.1 in the rules distribution; that would be too 
significant a value. I also think that there is value in having matched rules 
showing up in the long form (folded header) of the SA report with "0.0" if they 
are intended to have no direct impact on the ham/spam decision.


-- 
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

Fwd: Re: Rule: "1.0 R_DCD 90% of .com. is spam"

[Benny Pedersen]

oh dear, when do he stop ?

 Original besked 
Emne: Re: Rule: "1.0 R_DCD 90% of .com. is spam"
Dato: 2024-05-10 20:17
Afsender: "Reindl Harald (gmail)" 
Modtager: Benny Pedersen 

Am 10.05.24 um 20:14 schrieb Benny Pedersen:

Matus UHLAR - fantomas skrev den 2024-05-10 18:46:

On 10.05.24 15:36, Rupert Gallagher wrote:
The ikea mail was received through ... 
mta-numbers.ikea.com.sparkpostmail.com and is a request for feedback.


The SA rule says ...

header R_DCD Received =~ /\.com\./

I still do not know where the rule comes from, DCD may actually mean 
dot-com-dot, and perhaps it is true that they are mostly spam.


where is the rule stored? what file?


On May 10, 2024, 17:18, Rupert Gallagher wrote:
I only have stock and KAM, and it is definitely not a custom rule of 
mine.


grep -r '\.com./' /var/lib/spamassassin/4.00/

seems some good dot.com rules everwhere


and what has this to do with the other idiot?
go and eat shit you dumb list spammer

Re: Score 0.001

[Bill Cole]

On 2024-05-10 at 11:00:45 UTC-0400 (Fri, 10 May 2024 08:00:45 -0700 (PDT))
John Hardin 
is rumored to have said:

> Note that poorly-performing rules may get a score that looks informational, 
> but that may change over time based on the corpora.

IOW: rules that in themselves are not good enough performers to get included in 
the daily active list will still be pulled into the active list with a trivial 
score if derivative meta rules which are good enough for real scores depend on 
them.

-- 
Bill Cole