Re: SPF skipped for whitelisted relay domain

[Alex]

Hi,


> this is question for policyd-spf and its configuration.
>
> >The problem here is that something appears to be preventing my
> >welcomelist_auth entries from working properly, but I don't really
> >understand how.
>
> I guess it's the whitelist in policyd-spf.


Is it possible that it's somehow being passed through the port it uses to
communicate with postfix, and it's somehow using some postfix whitelist?

It most certainly isn't coming from a whitelist in policyd-spf, because it
happens even when it's completely removed.

I've also asked on the policyd-spf github page with no response.

Re: SPF skipped for whitelisted relay domain

[Matus UHLAR - fantomas]

>https://pastebin.com/TvTx6KzY

X-Comment: SPF skipped for whitelisted relay domain -
client-ip=13.110.6.221; helo=smtp14-ph2-sp4.mta.salesforce.com;
envelope-from=re...@support.meridianlink.com; receiver=
X-Greylist: whitelisted by SQLgrey-1.8.0

isn't it possible that it's sqlgrey that whitelisted your domain?


On 09.05.22 08:11, Alex wrote:

Yes, I suppose that's possible - meridianlink.com is listed in
my clients_fqdn_whitelist.local file, but how would policyd-spf interpret
that it should whitelist SPF? How would that communication even occur? That
"SPF skipped for whitelisted relay domain" content is coming from
policyd-spf.


this is question for policyd-spf and its configuration.


The problem here is that something appears to be preventing my
welcomelist_auth entries from working properly, but I don't really
understand how.


I guess it's the whitelist in policyd-spf. 


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
On the other hand, you have different fingers.

Re: SPF skipped for whitelisted relay domain

[Alex]

Hi,


> >https://pastebin.com/TvTx6KzY
>
> X-Comment: SPF skipped for whitelisted relay domain -
> client-ip=13.110.6.221; helo=smtp14-ph2-sp4.mta.salesforce.com;
> envelope-from=re...@support.meridianlink.com; receiver=
> X-Greylist: whitelisted by SQLgrey-1.8.0
>
> isn't it possible that it's sqlgrey that whitelisted your domain?
>

Yes, I suppose that's possible - meridianlink.com is listed in
my clients_fqdn_whitelist.local file, but how would policyd-spf interpret
that it should whitelist SPF? How would that communication even occur? That
"SPF skipped for whitelisted relay domain" content is coming from
policyd-spf.

The problem here is that something appears to be preventing my
welcomelist_auth entries from working properly, but I don't really
understand how.

Thanks so much for your help.

Re: SPF skipped for whitelisted relay domain

[Matus UHLAR - fantomas]

>I'm trying to understand why some domains are not whitelisted even
>though they pass SPF and are in my local welcomelist_auth entries. I'm
>using policyd-spf with postfix, and it appears to be adding the
>following header:
>
>X-Comment: SPF skipped for whitelisted relay domain -
>client-ip=13.110.6.221; helo=smtp14-ph2-sp4.mta.salesforce.com;
>envelope-from=re...@support.meridianlink.com; receiver=

you seem to have domain listed in whitelist policyd-spf whitelist.
salesforce.com probably?


On 07.05.22 13:29, Alex wrote:

I figured out where it's whitelisted, but still don't understand how it works.

It's somehow referencing the postscreen access list I'm using:

postscreen_access_list =
   permit_mynetworks, cidr:$config_directory/postscreen_access.cidr

In that file are cidr entries like:
13.110.208.0/21 permit
13.110.216.0/22 permit
13.110.224.0/20 permit


this is just postscreen whitelist, potscreen does not look up for SPF
unless something else uses this file somehow, this is not the problem.

I also sayt that the message says that "whitelisted relay domain", so it's 
apparently not the IP address but the domain that is whitelisted.


Still, 


I was aware of this access list, but I wasn't aware that the policy
daemon was also using it as well as postscreen.



The problem now is that I don't know _how_ it's using it, and how to
prevent it from affecting my welcomelist_auth entries. I don't see any
reference in the code that would indicate it's somehow getting this
info from postscreen/postfix and using it when making these decisions.

The unmodified original messages also no longer pass SPF - shouldn't
they? It does still pass DKIM from the command-line, and therefore my
welcomelist_auth entry, but not when it's first received.


you must search in policy daemon configuration and docs, this is not done by 
postfix.
 

There was a reason I added this email to the welcomelist in the first
place. Perhaps a temporary solution would be to just remove the
postscreen access lists for now? Other ideas? Someone would like to
help me troubleshoot this? I'm thinking the fact that the IP is
whitelisted in postscreen is somehow being passed through the socket
to policyd-spf in a structure somewhere.


I still have no idea who and how whitelists this sender, so I can't tell you 
what whitelist to remove.



>My welcomelist entry in SA for this specific email is as:
>welcomelist_auth re...@support.meridianlink.com

is this in spamassassin's local.cf ?


Yes


have you reloaded amavisd after you added it?



>salesforce is also listed in their SPF record:
>$ dig +short txt support.meridianlink.com
>"v=spf1 include:spf.protection.outlook.com include:_spf.salesforce.com -all"

SPF_PASS idicates that the SPF hit.

however, posting full headers could help us a bit.


https://pastebin.com/TvTx6KzY


X-Comment: SPF skipped for whitelisted relay domain - client-ip=13.110.6.221; helo=smtp14-ph2-sp4.mta.salesforce.com; envelope-from=re...@support.meridianlink.com; receiver= 
X-Greylist: whitelisted by SQLgrey-1.8.0


isn't it possible that it's sqlgrey that whitelisted your domain?


$ spamassassin --version
SpamAssassin version 4.0.0-r1889518
 running on Perl version 5.32.1


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
We are but packets in the Internet of life (userfriendly.org)

Re: SPF skipped for whitelisted relay domain

[Alex]

> >I'm trying to understand why some domains are not whitelisted even
> >though they pass SPF and are in my local welcomelist_auth entries. I'm
> >using policyd-spf with postfix, and it appears to be adding the
> >following header:
> >
> >X-Comment: SPF skipped for whitelisted relay domain -
> >client-ip=13.110.6.221; helo=smtp14-ph2-sp4.mta.salesforce.com;
> >envelope-from=re...@support.meridianlink.com; receiver=
>
> you seem to have domain listed in whitelist policyd-spf whitelist.
> salesforce.com probably?

I figured out where it's whitelisted, but still don't understand how it works.

It's somehow referencing the postscreen access list I'm using:

postscreen_access_list =
permit_mynetworks, cidr:$config_directory/postscreen_access.cidr

In that file are cidr entries like:
13.110.208.0/21 permit
13.110.216.0/22 permit
13.110.224.0/20 permit

This file is auto-generated from my postwhite script that gathers IPs
for the "too big to fail" providers like salesforce and google and
microsoft.

which match the client IP for salesforce:
client-ip=13.110.6.221; helo=smtp14-ph2-sp4.mta.salesforce.com

I was aware of this access list, but I wasn't aware that the policy
daemon was also using it as well as postscreen.

The problem now is that I don't know _how_ it's using it, and how to
prevent it from affecting my welcomelist_auth entries. I don't see any
reference in the code that would indicate it's somehow getting this
info from postscreen/postfix and using it when making these decisions.

The unmodified original messages also no longer pass SPF - shouldn't
they? It does still pass DKIM from the command-line, and therefore my
welcomelist_auth entry, but not when it's first received.

There was a reason I added this email to the welcomelist in the first
place. Perhaps a temporary solution would be to just remove the
postscreen access lists for now? Other ideas? Someone would like to
help me troubleshoot this? I'm thinking the fact that the IP is
whitelisted in postscreen is somehow being passed through the socket
to policyd-spf in a structure somewhere.

> >My welcomelist entry in SA for this specific email is as:
> >welcomelist_auth re...@support.meridianlink.com
>
> is this in spamassassin's local.cf ?

Yes

> >salesforce is also listed in their SPF record:
> >$ dig +short txt support.meridianlink.com
> >"v=spf1 include:spf.protection.outlook.com include:_spf.salesforce.com -all"
>
> SPF_PASS idicates that the SPF hit.
>
> however, posting full headers could help us a bit.

https://pastebin.com/TvTx6KzY

$ spamassassin --version
SpamAssassin version 4.0.0-r1889518
  running on Perl version 5.32.1

Re: SPF skipped for whitelisted relay domain

[Kevin A. McGrail]

> we wait for spamassassin 4.0.0 :=)
>
> 4.0.0  is in pre-release now and in production for a few of us.  Start
stress testing it now so we can shake out the bugs and get it out the door!

Regards,
KAM

Re: SPF skipped for whitelisted relay domain

[Benny Pedersen]

On 2022-05-06 05:35, Kevin A. McGrail wrote:

Hi Alex, sometimes I see this when the envelope from doesn't match the
header from. So what you think might pass SPF does not. That's my only
guess from looking at the example you posted. That example looked like
it would work perfectly.


we wait for spamassassin 4.0.0 :=)

amavisd does not know anything about spf btw


X-Comment: SPF skipped for whitelisted relay domain -
client-ip=13.110.6.221; helo=smtp14-ph2-sp4.mta.salesforce.com [1];
envelope-from=re...@support.meridianlink.com; receiver=


mail::spf does not know this header, its seen as no spf is done

set pypolicyspf to add A-R headers

Re: SPF skipped for whitelisted relay domain

[Matus UHLAR - fantomas]

On 05.05.22 18:01, Alex wrote:

I'm trying to understand why some domains are not whitelisted even
though they pass SPF and are in my local welcomelist_auth entries. I'm
using policyd-spf with postfix, and it appears to be adding the
following header:

X-Comment: SPF skipped for whitelisted relay domain -
client-ip=13.110.6.221; helo=smtp14-ph2-sp4.mta.salesforce.com;
envelope-from=re...@support.meridianlink.com; receiver=


you seem to have domain listed in whitelist policyd-spf whitelist.
salesforce.com probably?

I'm not sure if this is needed, policyd-spf could add Received-SPF: header 
that SA could use (and avoid duplicate lookups)



I realize this may not necessarily be directly related to SA, but it's
apparently affecting my ability to process SPF headers with
amavisd/SA, and I hoped someone could help.

What's happening where the mail passes SPF but still bypasses my
welcomelist entries? My skip_addresses list doesn't include this
particular IP:
skip_addresses =
139.138.56.0/24,127.0.0.0/8,:::127.0.0.0/104,::1,52.128.98.0/24,74.203.184.0/24,74.200.60.0/24,209.222.82.0/24,12.15.90.10


My welcomelist entry in SA for this specific email is as:
welcomelist_auth re...@support.meridianlink.com


is this in spamassassin's local.cf ?


The amavisd headers show it passed SPF:

Return-Path: 
X-Spam-Status: No, score=-2.491 tagged_above=-200 required=5
   tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
   DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, EXTRACTTEXT=0.001,
   FMBLA_HELO_OUTMX=-0.01, FMBLA_RDNS_OUTMX=-0.01,
   HTML_IMAGE_RATIO_08=0.001, HTML_MESSAGE=0.001, LOC_CDIS_INLINE=0.1,
   LOC_IMGSPAM=0.1, RCVD_IN_DNSWL_NONE=-0.0001,
   RCVD_IN_SENDERSCORE_90_100=-0.6, RELAYCOUNTRY_US=0.01,
   SPF_HELO_NONE=0.001, SPF_PASS=-0.001, TXREP=0.016] autolearn=disabled

This one didn't need to be added to the welcomelist, but others do.
The last header received before reaching our server is as:

Received: from smtp14-ph2-sp4.mta.salesforce.com
(smtp14-ph2-sp4.mta.salesforce.com [13.110.6.221])
   (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
   (No client certificate requested)
   by mail01.example.com (Postfix) with ESMTPS id 5FC7010024E93
   for ; Thu,  5 May 2022 12:01:59 -0400 (EDT)

salesforce is also listed in their SPF record:
$ dig +short txt support.meridianlink.com
"v=spf1 include:spf.protection.outlook.com include:_spf.salesforce.com -all"


SPF_PASS idicates that the SPF hit. 


however, posting full headers could help us a bit.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
"They say when you play that M$ CD backward you can hear satanic messages."
"That's nothing. If you play it forward it will install Windows."

Re: SPF skipped for whitelisted relay domain

[Kevin A. McGrail]

Hi Alex, sometimes I see this when the envelope from doesn't match the
header from. So what you think might pass SPF does not. That's my only
guess from looking at the example you posted. That example looked like it
would work perfectly.  KAM

On Thu, May 5, 2022, 18:02 Alex  wrote:

> Hi,
>
> I'm trying to understand why some domains are not whitelisted even
> though they pass SPF and are in my local welcomelist_auth entries. I'm
> using policyd-spf with postfix, and it appears to be adding the
> following header:
>
> X-Comment: SPF skipped for whitelisted relay domain -
> client-ip=13.110.6.221; helo=smtp14-ph2-sp4.mta.salesforce.com;
> envelope-from=re...@support.meridianlink.com; receiver=
>
> I realize this may not necessarily be directly related to SA, but it's
> apparently affecting my ability to process SPF headers with
> amavisd/SA, and I hoped someone could help.
>
> What's happening where the mail passes SPF but still bypasses my
> welcomelist entries? My skip_addresses list doesn't include this
> particular IP:
> skip_addresses =
>
> 139.138.56.0/24,127.0.0.0/8,:::127.0.0.0/104,::1,52.128.98.0/24,74.203.184.0/24,74.200.60.0/24,209.222.82.0/24,12.15.90.10
>
>
> My welcomelist entry in SA for this specific email is as:
> welcomelist_auth re...@support.meridianlink.com
>
> The amavisd headers show it passed SPF:
>
> Return-Path: 
> X-Spam-Status: No, score=-2.491 tagged_above=-200 required=5
> tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
> DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, EXTRACTTEXT=0.001,
> FMBLA_HELO_OUTMX=-0.01, FMBLA_RDNS_OUTMX=-0.01,
> HTML_IMAGE_RATIO_08=0.001, HTML_MESSAGE=0.001, LOC_CDIS_INLINE=0.1,
> LOC_IMGSPAM=0.1, RCVD_IN_DNSWL_NONE=-0.0001,
> RCVD_IN_SENDERSCORE_90_100=-0.6, RELAYCOUNTRY_US=0.01,
> SPF_HELO_NONE=0.001, SPF_PASS=-0.001, TXREP=0.016] autolearn=disabled
>
> This one didn't need to be added to the welcomelist, but others do.
> The last header received before reaching our server is as:
>
> Received: from smtp14-ph2-sp4.mta.salesforce.com
> (smtp14-ph2-sp4.mta.salesforce.com [13.110.6.221])
> (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
> (No client certificate requested)
> by mail01.example.com (Postfix) with ESMTPS id 5FC7010024E93
> for ; Thu,  5 May 2022 12:01:59 -0400 (EDT)
>
> salesforce is also listed in their SPF record:
> $ dig +short txt support.meridianlink.com
> "v=spf1 include:spf.protection.outlook.com include:_spf.salesforce.com
> -all"
>
> Thanks,
> Alex
>

Re: spf fails at apache.org forwards ipv6

[Matus UHLAR - fantomas]

Benny Pedersen:

: host
   mx1-he-de.apache.org[2a01:4f8:c2c:2bf7::1] said: 550 5.7.23
   : Recipient address rejected: 
ASF gnomes

   rejected your message: SPF fail - not authorized. See
   https://infra.apache.org/mail-rejection.html (in reply to RCPT TO
command)


is it solved ?



On 2022-01-19 11:41, David Bürgin wrote:

Impossible to say more without knowing the context (sender email and IP
address).


On 19.01.22 16:02, Benny Pedersen wrote:

my own flatted ips is

v=spf1 ip4:172.104.150.56 ip6:2a01:7e01::f03c:92ff:fe3b:151e 
ip6:2a01:7e01:e001:289::1 ip6:2a01:7e01:e001:289::2/127 
ip6:2a01:7e01:e001:289::4 -all



perhaps Received: headers from the mail you have received.
If that mail was rejected within apache network, you should see which server
rejected from which one.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
99 percent of lawyers give the rest a bad name.

Re: spf fails at apache.org forwards ipv6

[Benny Pedersen]

On 2022-01-19 11:41, David Bürgin wrote:

Benny Pedersen:

: host
mx1-he-de.apache.org[2a01:4f8:c2c:2bf7::1] said: 550 5.7.23
: Recipient address rejected: ASF 
gnomes

rejected your message: SPF fail - not authorized. See
https://infra.apache.org/mail-rejection.html (in reply to RCPT TO
command)


is it solved ?


The server rejected your message because you are using a sender address
that is not allowed according to SPF policy?


spf enveloppe changes on next server and it was not accepted internal

v=spf1 ip4:3.227.148.255 ip4:95.216.194.37 ip4:116.203.82.107 
ip4:116.203.166.180 ip4:159.69.187.90 ip4:198.2.128.0/24 
ip4:198.2.132.0/22 ip4:198.2.136.0/23 ip4:198.2.145.0/24 
ip4:198.2.177.0/24 ip4:198.2.178.0/23 ip4:198.2.180.0/24 
ip4:198.2.186.0/23 ip4:205.201.131.128/25 ip4:205.201.134.128/25 
ip4:205.201.136.0/23 ip4:205.201.139.0/24 ip4:207.244.88.131 
ip4:207.244.88.144 ip4:207.244.88.153 ip6:2a01:4f8:c2c:e8b::/64 
ip6:2a01:4f9:c010:567c::1 -all


so one hetzner server was not accepted on apache.org content filters ?

i think the content filter part did not change envelope sender before 
checked spf


i should not speculate, but its common error if more then 256 mx ips in 
ipv4, have not counted ipv6 yet



Impossible to say more without knowing the context (sender email and IP
address).


my own flatted ips is

v=spf1 ip4:172.104.150.56 ip6:2a01:7e01::f03c:92ff:fe3b:151e 
ip6:2a01:7e01:e001:289::1 ip6:2a01:7e01:e001:289::2/127 
ip6:2a01:7e01:e001:289::4 -all

Re: spf fails at apache.org forwards ipv6

[David Bürgin]

Benny Pedersen:
> : host
> mx1-he-de.apache.org[2a01:4f8:c2c:2bf7::1] said: 550 5.7.23
> : Recipient address rejected: ASF gnomes
> rejected your message: SPF fail - not authorized. See
> https://infra.apache.org/mail-rejection.html (in reply to RCPT TO
> command)
> 
> 
> is it solved ?

The server rejected your message because you are using a sender address
that is not allowed according to SPF policy?

Impossible to say more without knowing the context (sender email and IP
address).

Re: SPF plugin ignores existing Authentication-Results

[David Bürgin]

Matus UHLAR - fantomas:

Matus UHLAR - fantomas:

this is more an issue of how milter itself operates.

the milter is supposed to see e-mail as it was received from (smtp) client -
even without Received: headers, just with other milters' modifications.

If SpamAssassin (SA from now) has to see Authentication-Results: headers
from other milter, the other milter must run before milter using SA
(spamass-milter, amavisd-milter, etc)

SA milter has to synthetize the Received: header it passsed with mail to
SpamAssassin.  If it prepends Received: header (as expected), the
Authentication-Results: header(s) added by the former milter appear after
Received: and SA doesn't trust it.

...nobody sees the synthetized Received: header later, they see Received:
added by MTA, before Authentication-Results added by mentioned milter.


On 18.05.21 14:12, David Bürgin wrote:

Thank you, this is a good summary of the problem.


I want to add to this old thread, that last few changes fixed this problem.

When milter adds header, it can add it at position 0 which means before
MTA-added Received: header.

pull requests were submitted to put Authentication-Results in case of
OpenDKIM - https://github.com/trusteddomainproject/OpenDKIM/pull/126
OpenDMARC - https://github.com/trusteddomainproject/OpenDMARC/pull/171
and openARC - https://github.com/trusteddomainproject/OpenARC/pull/141

so hopefully, next releases will put headers at proper place.

I have patched OpenDKIM and OpenDMARC and can verify, that not only other
milters, but also later programs see Authentication-Results: at proper
place, so sendmail can trust them.


Just to be clear, these changes don’t help in the SpamAssassin milter
case. A workaround like ‘--synth-relay’ proposed in this thread is still
necessary when SpamAssassin is integrated as a milter.

Re: SPF plugin ignores existing Authentication-Results

[Matus UHLAR - fantomas]

Matus UHLAR - fantomas:

this is more an issue of how milter itself operates.

the milter is supposed to see e-mail as it was received from (smtp) client -
even without Received: headers, just with other milters' modifications.

If SpamAssassin (SA from now) has to see Authentication-Results: headers
from other milter, the other milter must run before milter using SA
(spamass-milter, amavisd-milter, etc)

SA milter has to synthetize the Received: header it passsed with mail to
SpamAssassin.  If it prepends Received: header (as expected), the
Authentication-Results: header(s) added by the former milter appear after
Received: and SA doesn't trust it.

...nobody sees the synthetized Received: header later, they see Received:
added by MTA, before Authentication-Results added by mentioned milter.


On 18.05.21 14:12, David Bürgin wrote:

Thank you, this is a good summary of the problem.


I want to add to this old thread, that last few changes fixed this problem.

When milter adds header, it can add it at position 0 which means before
MTA-added Received: header.

pull requests were submitted to put Authentication-Results in case of
OpenDKIM - https://github.com/trusteddomainproject/OpenDKIM/pull/126
OpenDMARC - https://github.com/trusteddomainproject/OpenDMARC/pull/171
and openARC - https://github.com/trusteddomainproject/OpenARC/pull/141

so hopefully, next releases will put headers at proper place.

I have patched OpenDKIM and OpenDMARC and can verify, that not only other
milters, but also later programs see Authentication-Results: at proper
place, so sendmail can trust them.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Saving Private Ryan...
Private Ryan exists. Overwrite? (Y/N)

Re: SPF plugin ignores existing Authentication-Results

[David Bürgin]

David Bürgin:
> David Bürgin:
> > Bother. I think I will try to modify my SpamAssassin milter, so that it
> > will add a synthetic ‘internal’ Received header right after the
> > Authentication-Results headers … that should trick SpamAssassin into
> > recognising them as internal.
> 
> Here’s the plan to address this in SpamAssassin Milter
> (https://crates.io/crates/spamassassin-milter): Add an option that takes
> a regular expression which matches headers *after which* the synthetic
> ‘internal’ Received header should be inserted.

For those following along, I have added a new experimental option to my
SpamAssassin milter implementation. Here’s the documentation:

   --synth-relay POS
  Synthesize  the internal relay header after position POS.  POS may
  be either an integer, in which case the relay header will  be  in‐
  serted  after  skipping POS header lines, or a regular expression,
  in which case the relay header will be inserted after skipping all
  header  lines  matching  POS.  This option is useful in situations
  where the default  position  of  the  synthesized  internal  relay
  header  (the  MTA’s Received header, https://cwiki.apache.org/con‐
  fluence/display/SPAMASSASSIN/TrustedRelays) at the very  beginning
  of the message header is not appropriate.  For example, when other
  milters insert Authentication-Results headers for SpamAssassin  to
  consume, SpamAssassin will not trust these headers unless they ap‐
  pear before the internal relay header; in this case setting POS to
  “^(?i)authentication-results:”  achieves  proper  placement of the
  relay header after such header lines.

Now, using the following setting I should get SpamAssassin to reuse
Authentication-Results provided by SPF Milter:

--synth-relay '^(?i)authentication-results: *mail\.gluet\.ch;'

And here we go:

May 21 19:08:21 mail spf-milter[42418]: mxout1-he-de.apache.org (helo): pass
May 21 19:08:21 mail spf-milter[42418]: 
users-return-124577-dbuergin=gluet...@spamassassin.apache.org (mailfrom): pass
May 21 19:08:21 mail spf-milter[42418]: E87B4400D309: adding 
Authentication-Results header
May 21 19:08:21 mail spf-milter[42418]: E87B4400D309: Authentication-Results: 
mail.gluet.ch; spf=pass smtp.helo=mxout1-he-de.apache.org
May 21 19:08:21 mail spf-milter[42418]: E87B4400D309: adding 
Authentication-Results header
May 21 19:08:21 mail spf-milter[42418]: E87B4400D309: Authentication-Results: 
mail.gluet.ch; spf=pass smtp.mailfrom=spamassassin.apache.org
May 21 19:08:22 mail spamassassin-milter[55644]: E87B4400D309: inserting 
synthetic relay header at index 4
May 21 19:08:22 mail spamd[56256]: spf: checking to see if the message has a 
Received-SPF header that we can use
May 21 19:08:22 mail spamd[56256]: spf: found an Authentication-Results header 
added by an internal host: Authentication-Results: mail.gluet.ch; spf=pass 
smtp.helo=mxout1-he-de.apache.org
May 21 19:08:22 mail spamd[56256]: spf: re-using helo result from 
Authentication-Results header: pass
May 21 19:08:22 mail spamd[56256]: spf: found an Authentication-Results header 
added by an internal host: Authentication-Results: mail.gluet.ch; spf=pass 
smtp.mailfrom=spamassassin.apache.org
May 21 19:08:22 mail spamd[56256]: spf: re-using mfrom result from 
Authentication-Results header: pass

Success! For now I’m keeping this on a separate branch. Don’t know if it
feels too hacky to keep, feedback welcome. I would prefer if this could
be solved in SpamAssassin (bug 6918).

Ciao,
David

Re: SPF plugin ignores existing Authentication-Results

[Matus UHLAR - fantomas]

Matus UHLAR - fantomas:

Possible workarounds require trusting the Authentication-Results: header
either via SA milter (which would add synthetized Received: header after
it), or via SpamAssassin itself (trust headers added by "host" immediately
after last trusted/internal "Received" header.)

I prefer the second setup, as it's possible to re-check SA score for such
e-mail later.

I have tried receiving mail with fake Authentication-Results: header and it
got deleted by opendkim-milter, to opendkim-milter may be trusted for this
setup.

SA would need an option which hosts to trust Authentication-Results: from.


On 18.05.21 14:12, David Bürgin wrote:

To the SpamAssassin developers: Would you consider adding a
configuration option to trust *all* Authentication-Results headers with
a certain authserv-id
(https://www.rfc-editor.org/rfc/rfc8601#section-2.5)? That would be very
helpful for setups that (securely) manage Authentication-Results headers
themselves.


https://bz.apache.org/SpamAssassin/show_bug.cgi?id=6918

I'll add the info there.


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Emacs is a complicated operating system without good text editor.

Re: SPF plugin ignores existing Authentication-Results

[David Bürgin]

Matus UHLAR - fantomas:

this is more an issue of how milter itself operates.

the milter is supposed to see e-mail as it was received from (smtp) client -
even without Received: headers, just with other milters' modifications.

If SpamAssassin (SA from now) has to see Authentication-Results: headers
from other milter, the other milter must run before milter using SA
(spamass-milter, amavisd-milter, etc)

SA milter has to synthetize the Received: header it passsed with mail to
SpamAssassin.  If it prepends Received: header (as expected), the
Authentication-Results: header(s) added by the former milter appear after
Received: and SA doesn't trust it.

...nobody sees the synthetized Received: header later, they see Received:
added by MTA, before Authentication-Results added by mentioned milter.


Thank you, this is a good summary of the problem.


Possible workarounds require trusting the Authentication-Results: header
either via SA milter (which would add synthetized Received: header after
it), or via SpamAssassin itself (trust headers added by "host" immediately
after last trusted/internal "Received" header.)

I prefer the second setup, as it's possible to re-check SA score for such
e-mail later.

I have tried receiving mail with fake Authentication-Results: header and it
got deleted by opendkim-milter, to opendkim-milter may be trusted for this
setup.

SA would need an option which hosts to trust Authentication-Results: from.


To the SpamAssassin developers: Would you consider adding a
configuration option to trust *all* Authentication-Results headers with
a certain authserv-id
(https://www.rfc-editor.org/rfc/rfc8601#section-2.5)? That would be very
helpful for setups that (securely) manage Authentication-Results headers
themselves.

Re: SPF plugin ignores existing Authentication-Results

[Matus UHLAR - fantomas]

Martin Gregorie:

Have you set the 'internal_networks' configuration parameter (in
local.cf)? If not, try that first.


On 18.05.21 12:07, David Bürgin wrote:

Thanks, but I don’t think this helps here. I don’t know what I could add
to internal_networks that would somehow change the behaviour. The
problem is with how milters for SpamAssassin operate.


this is more an issue of how milter itself operates.

the milter is supposed to see e-mail as it was received from (smtp) client -
even without Received: headers, just with other milters' modifications.

If SpamAssassin (SA from now) has to see Authentication-Results: headers
from other milter, the other milter must run before milter using SA
(spamass-milter, amavisd-milter, etc)

SA milter has to synthetize the Received: header it passsed with mail to
SpamAssassin.  If it prepends Received: header (as expected), the
Authentication-Results: header(s) added by the former milter appear after
Received: and SA doesn't trust it.

...nobody sees the synthetized Received: header later, they see Received:
added by MTA, before Authentication-Results added by mentioned milter.


Possible workarounds require trusting the Authentication-Results: header
either via SA milter (which would add synthetized Received: header after
it), or via SpamAssassin itself (trust headers added by "host" immediately
after last trusted/internal "Received" header.)

I prefer the second setup, as it's possible to re-check SA score for such
e-mail later.

I have tried receiving mail with fake Authentication-Results: header and it
got deleted by opendkim-milter, to opendkim-milter may be trusted for this
setup.

SA would need an option which hosts to trust Authentication-Results: from.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
BSE = Mad Cow Desease ... BSA = Mad Software Producents Desease

Re: SPF plugin ignores existing Authentication-Results

[David Bürgin]

Martin Gregorie:

Have you set the 'internal_networks' configuration parameter (in
local.cf)? If not, try that first.


Thanks, but I don’t think this helps here. I don’t know what I could add
to internal_networks that would somehow change the behaviour. The
problem is with how milters for SpamAssassin operate.

Re: SPF plugin ignores existing Authentication-Results

[Martin Gregorie]

On Tue, 2021-05-18 at 10:00 +0200, David Bürgin wrote:
> David Bürgin:
> > Bother. I think I will try to modify my SpamAssassin milter, so that
> > it
> > will add a synthetic ‘internal’ Received header right after the
> > Authentication-Results headers … that should trick SpamAssassin into
> > recognising them as internal.
> 
Have you set the 'internal_networks' configuration parameter (in
local.cf)? If not, try that first.

Martin

Re: SPF plugin ignores existing Authentication-Results

[David Bürgin]

David Bürgin:

Bother. I think I will try to modify my SpamAssassin milter, so that it
will add a synthetic ‘internal’ Received header right after the
Authentication-Results headers … that should trick SpamAssassin into
recognising them as internal.


Here’s the plan to address this in SpamAssassin Milter
(https://crates.io/crates/spamassassin-milter): Add an option that takes
a regular expression which matches headers *after which* the synthetic
‘internal’ Received header should be inserted.

--trusted-relay-after 

Then one would use this like this:

--trusted-relay-after '^(?i)authentication-results:[\t ]*mail\.gluet\.ch;'

The effect this would have is that a message that is sent to
SpamAssassin in this form currently:

Received: from m43-8.mailgun.net by mail.gluet.ch (Postfix) ...
Authentication-Results: mail.gluet.ch; dmarc=pass ...  [← untrusted!]
Authentication-Results: mail.gluet.ch; dkim=pass ...   [← untrusted!]
Authentication-Results: mail.gluet.ch; spf=pass ...[← untrusted!]
Message-ID: <1...@execute-api.eu-central-1.amazonaws.com>
...

Would now be sent to SpamAssassin in this form:

Authentication-Results: mail.gluet.ch; dmarc=pass ...  [← now trusted]
Authentication-Results: mail.gluet.ch; dkim=pass ...   [← now trusted]
Authentication-Results: mail.gluet.ch; spf=pass ...[← now trusted]
Received: from m43-8.mailgun.net by mail.gluet.ch (Postfix) ...
Message-ID: <1...@execute-api.eu-central-1.amazonaws.com>
...

I’m interested to hear what comments others might have about the problem
and this solution. (Also a bit surprised that no one seems to have
noticed this before.)

To highlight the problem once more: When used as a milter, SpamAssassin
cannot take into account Authentication-Results added by earlier
milters, because it does not trust them.

Re: SPF plugin ignores existing Authentication-Results

[David Bürgin]

David Bürgin:

I remember asking here if SpamAssassin is able to use these instead of
doing its own SPF queries. Now with debug logging on, I see that
SpamAssassin isn’t actually using these results:

 ...

Is this a bug in the SPF plugin? Do I need to set something in my
config? I’m using SpamAssassin 3.4.4-1ubuntu1.1 on Ubuntu Server 20.04.


Oh, it actually cannot work. SpamAssassin only accepts
Authentication-Results that come *before* an ‘internal’ Received header
(https://cwiki.apache.org/confluence/display/SPAMASSASSIN/TrustedRelays).
I use SpamAssassin as a milter, so the ‘internal’ Received header is
added by Postfix only *after* the milters add their headers. That means
my milters’ Authentication-Results headers will never look like internal
to SpamAssassin, and will never be used.

Bother. I think I will try to modify my SpamAssassin milter, so that it
will add a synthetic ‘internal’ Received header right after the
Authentication-Results headers … that should trick SpamAssassin into
recognising them as internal.

Re: spf fail !

[Benny Pedersen]

Noel Butler skrev den 2020-07-19 04:12:


you'll find other mail rejected as well, since someone changed from
hermes to this mailroute machine...


X-Spam-Rules_score: 
HEADER_FROM_DIFFERENT_DOMAINS=0.001,HTML_MESSAGE=0.1,

MAILING_LIST_MULTI=-10.1,RCVD_IN_DNSWL_NONE=-0.1,SPF_FAIL=6.419,
SPF_HELO_NONE=2.1,TXREP=-0.673,UNPARSEABLE_RELAY=0.001,
USER_IN_WHITELIST=-1


infra didnt mention anything about the change


drovned in chocolate :=)

Re: spf fail !

[Kevin A. McGrail]

On 7/18/2020 10:12 PM, Noel Butler wrote:
>
> On 19/07/2020 10:45, Benny Pedersen wrote:
>
>> Received: from mailroute1-lw-us.apache.org
>> (mailroute1-lw-us.apache.org [207.244.88.153])
>> by mx.junc.eu (Postfix) with ESMTPS
>> for > >; Sun, 19 Jul 2020 00:41:16 + (UTC)
>>
>> gives spf fails now, be carefull
>>
>>
> you'll find other mail rejected as well, since someone changed from
> hermes to this mailroute machine...
>
> infra didnt mention anything about the change
>
Thanks, I've email Infra.  Appreciate the heads-up.

-- 
Kevin A. McGrail
kmcgr...@apache.org

Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Re: spf fail !

[Noel Butler]

On 19/07/2020 10:45, Benny Pedersen wrote:

> Received: from mailroute1-lw-us.apache.org (mailroute1-lw-us.apache.org 
> [207.244.88.153])
> by mx.junc.eu (Postfix) with ESMTPS
> for ; Sun, 19 Jul 2020 00:41:16 + (UTC)
> 
> gives spf fails now, be carefull

you'll find other mail rejected as well, since someone changed from
hermes to this mailroute machine... 

infra didnt mention anything about the change

-- 
Regards,
Noel Butler 

This Email, including attachments, may contain legally privileged
information, therefore at all times remains confidential and subject to
copyright protected under international law. You may not disseminate
this message without the authors express written authority to do so. If
you are not the intended recipient, please notify the sender then delete
all copies of this message including attachments immediately.
Confidentiality, copyright, and legal privilege are not waived or lost
by reason of the mistaken delivery of this message.

Re: spf none and dkim not pass domains

[Benny Pedersen]

Kevin A. McGrail skrev den 2019-08-27 01:40:

I believe you will find lazy domain security rules in KAM.cf that can
help with this.  ?all, for example, is lazy SPF.


added whole kam.cf now, one problem solved another created :=)

Aug 28 00:05:23.366 [7470] info: rules: meta test JMQ_CONGRAT has 
dependency 'KAM_RAPTOR_ALTERED' with a zero score
Aug 28 00:05:23.371 [7470] info: rules: meta test KAM_JURY has 
dependency 'KAM_RAPTOR_ALTERED' with a zero score
Aug 28 00:05:23.379 [7470] info: rules: meta test KAM_BADPDF2 has 
dependency 'KAM_RPTR_SUSPECT' with a zero score
Aug 28 00:05:23.382 [7470] info: rules: meta test KAM_INSURE has 
dependency 'CBJ_GiveMeABreak' with a zero score
Aug 28 00:05:23.384 [7470] info: rules: meta test 
KAM_REALLY_FAKE_DELIVER has dependency 'KAM_RPTR_PASSED' with a zero 
score
Aug 28 00:05:23.385 [7470] info: rules: meta test KAM_CARD has 
dependency 'KAM_RPTR_SUSPECT' with a zero score
Aug 28 00:05:23.386 [7470] info: rules: meta test KAM_WARRANTY3 has 
dependency 'CBJ_GiveMeABreak' with a zero score
Aug 28 00:05:23.386 [7470] info: rules: meta test KAM_AUTO has 
dependency 'CBJ_GiveMeABreak' with a zero score
Aug 28 00:05:23.389 [7470] info: rules: meta test KAM_FAKE_DELIVER has 
dependency 'KAM_RAPTOR_ALTERED' with a zero score
Aug 28 00:05:23.391 [7470] info: rules: meta test KAM_INSURE2 has 
dependency 'CBJ_GiveMeABreak' with a zero score
Aug 28 00:05:23.394 [7470] info: rules: meta test KAM_NOTIFY2 has 
dependency 'KAM_IFRAME' with a zero score
Aug 28 00:05:23.395 [7470] info: rules: meta test KAM_WARRANTY has 
dependency 'CBJ_GiveMeABreak' with a zero score

Re: spf none and dkim not pass domains

[hg user]

Is it the spam coming as a empty subject, empty message and a pdf
attachment ?
I received about 3000 of them in the weekend and I'm starting to check the
logs of yesterday.
A lot of them got an high score, from 8 to 13 thanks to RBL...
score=9.692 required=5.6 tests=[BAYES_60=1.5, MY_RULE_1=-0.001,
FORGED_MUA_MOZILLA=2.309, KAM_LAZY_DOMAIN_SECURITY=1, KAM_MANYTO=0.2,
RCVD_IN_BL_SPAMCOP_NET=1.347, RCVD_IN_SBL_CSS=3.335, SPF_HELO_NONE=0.001,
SPF_NONE=0.001] autolearn=disabled

Yesterday I loaded some of them in the bayes engine and the results are
very, how can I say, "strange" and I may also add "dangerous"...

>From the info of the debug I'm testing a rule that checks for empty text,
empty body, pdf attachment, since, unfortunately, not all the sender
servers are in RBL...

Aug 27 08:08:20.993 [29203] dbg: check:
subtests=__ANY_TEXT_ATTACH,__ANY_TEXT_ATTACH_DOC,__BOTH_INR_AND_REF,...,,__CT,
__CTYPE_HAS_BOUNDARY,__CTYPE_MULTIPART_ANY,__CTYPE_MULTIPART_MIXED,__DKIM_DEPENDABLE,
__DOS_RCVD_MON,__DOS_RCVD_SUN,__DOS_RELAYED_EXT,__EMPTY_BODY,__ENV_AND_HDR_FROM_MATCH,
__FORGED_SENDER,__HAS_DATE,__HAS_FROM,__HAS_MESSAGE_ID,__HAS_MSGID,__HAS_RCVD,
__HAS_SUBJECT,__HAS_TO,__HAS_UA,__HDRS_LCASE_KNOWN,__KAM_DROPBOX2,__KAM_FAKEDELIVER12,
__KAM_FAKEDELIVER4,__KAM_FAKEDELIVER6,__KAM_FAKEDELIVER8,__KAM_GOOGLE2_2,
__KAM_HARP3,__KAM_HAS_0_URIS,__KAM_JURY3,__KAM_MAILSPLOIT2,__KAM_MANYTO,
__KAM_MANYTO,__KAM_MANYTO,__KAM_MANYTO,__KAM_MANYTO,__KAM_MANYTO2,
__KAM_MANYTO2,__KAM_MANYTO2,__KAM_MANYTO2,__KAM_MANYTO2,__KAM_MANYTO2,
__KAM_MANYTO2,__KAM_MULTIPLE_FROM,__KAM_PAYPAL3B,__KAM_SPF_NONE,
__KAM_UPS2,__KAM_URIBL_PCCC,__KAM_WU1,__KB_WAM_FROM_NAME_SINGLEWORD,
__KHOP_NO_FULL_NAME,__LAST_EXTERNAL_RELAY_NO_AUTH,__LAST_UNTRUSTED_RELAY_NO_AUTH,
__LCL__ENV_AND_HDR_FROM_MATCH,__MIME_ATTACHMENT,__MIME_BASE64,__MIME_VERSION,
__MISSING_REF,__MISSING_REPLY,__MOZILLA_MUA,__MSGID_APPLEMAIL,__MSGID_GUID,
__MSGID_OK_HOST,__MUA_TBIRD,__PART_STOCK_CD_F,__PDF_ATTACH,
__PDF_ATTACH_FN1,__PDF_ATTACH_FN2,__PDF_ATTACH_MT,__RCVD_IN_ZEN,
__RDNS_SHORT,__RP_MATCHES_RCVD,__SANE_MSGID,__SUBJECT_EMPTY,__SUBJ_SHORT,
__TOCC_EXISTS,__TVD_MIME_ATT_AP,__TVD_MIME_ATT_TP,__UA_MOZ5,__UNPARSEABLE_RELAY_COUNT


On Tue, Aug 27, 2019 at 1:40 AM Kevin A. McGrail 
wrote:

> I believe you will find lazy domain security rules in KAM.cf that can
> help with this.  ?all, for example, is lazy SPF.
>
> On 8/26/2019 19:20, Benny Pedersen wrote:
> > i see that bitcoins phinshing is trying to make use of not dkim signed
> > and use domains without spf, sounds silly and maybe its just not
> > detected yet anywhre, problem is as example
> >
> > From: "nets kundeservice" 
> >
> > how to block this if its not dkim signed ?
> >
> > same for spf, is it just checking spf none ?, and score that with 100
> > so spamas-milter will reject it ?
> >
> > what does others do to signal its not accepted, i am begin to be very
> > tired of phising and spaming abusers
> >
> > current 2033 ipv4 addresses listed here localy for trying sasl auth on
> > port 25
>
> --
> Kevin A. McGrail
> kmcgr...@apache.org
>
> Member, Apache Software Foundation
> Chair Emeritus Apache SpamAssassin Project
> https://www.linkedin.com/in/kmcgrail - 703.798.0171
>
>

Re: spf none and dkim not pass domains

[Kevin A. McGrail]

I believe you will find lazy domain security rules in KAM.cf that can
help with this.  ?all, for example, is lazy SPF.

On 8/26/2019 19:20, Benny Pedersen wrote:
> i see that bitcoins phinshing is trying to make use of not dkim signed
> and use domains without spf, sounds silly and maybe its just not
> detected yet anywhre, problem is as example
>
> From: "nets kundeservice" 
>
> how to block this if its not dkim signed ?
>
> same for spf, is it just checking spf none ?, and score that with 100
> so spamas-milter will reject it ?
>
> what does others do to signal its not accepted, i am begin to be very
> tired of phising and spaming abusers
>
> current 2033 ipv4 addresses listed here localy for trying sasl auth on
> port 25

-- 
Kevin A. McGrail
kmcgr...@apache.org

Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Re: SPF Fail for Amazon mails, although mail headers say its a pass

[Matus UHLAR - fantomas]

On 06.06.19 00:59, MarcelM wrote:

Ahh... I see. So probably other headers are modified by the mail server as
well, and that is why SA's SPF check fails!


this is probably SPF check for ampel-24.de which fails when forwarded
locally.

as you can see, amazon SPF succeeds:

Received: from a1-14.smtp-out.eu-west-1.amazonses.com 
(a1-14.smtp-out.eu-west-1.amazonses.com [54.240.1.14])
   by ampel24.de (Postfix) with ESMTPS id 2A2441CD89929
   for ; Wed,  5 Jun 2019 14:00:35 +0200 (CEST)
Authentication-Results: ampel24.de;
   spf=pass (sender IP is 54.240.1.14) 
smtp.mailfrom=201906051200349bb96326a7864e6eb55a703df7c0p...@bounces.amazon.de 
smtp.helo=a1-14.smtp-out.eu-west-1.amazonses.com

however, the sender is rewritten to 

which seems to cause SPF fail, because mail was received from amazon.


Why would it do that ? I will read up on that.


the SRS itself is OK, but apparently should be used after mail leaves
the server.

I have to think about this a bit more.


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
2B|!2B, that's a question!

Re: SPF Fail for Amazon mails, although mail headers say its a pass

[RW]

On Thu, 6 Jun 2019 09:30:31 +0200
Matus UHLAR - fantomas wrote:

> On 05.06.19 23:06, MarcelM wrote:
> >These are the full headers. (Sorry, did not realise all emails get
> >redacted)
> >
> >https://pastebin.com/Z6hkL9hD  
> 
> the mails didn't get redacted. The headers are quoted even on the
> pastebin example.
> 
> >and the non-forwared mail header too:
> >
> >https://pastebin.com/WGM0aYrh  
> 
> this one looks better.
> 
> >Does not look like SRS (but a good readup, something learned again).
> >I really don't get this. The spf record for 'amazon.de' states :
> >v=spf1 include:amazon.com -all
> >So I understand the SPF fail, since there is no 'amazon.com' to be
> >found here... but the mail server obviously thinks it's valid, which
> >it probably is, at least it is legit and amazon should have
> >correctly configured mail servers - I hope...  
> 
> Return-Path:
> 
> 
> this is mail from ampel-24.de, not amazon.

I don't think so, there's only one meaningful received header:

Received: from a1-14.smtp-out.eu-west-1.amazonses.com...by ampel24.de

what seems to have happened is that SRS rewriting has been applied
between receiving the email and the SA scan.

Re: SPF Fail for Amazon mails, although mail headers say its a pass

[Matus UHLAR - fantomas]

On 06.06.19 00:59, MarcelM wrote:

Ahh... I see. So probably other headers are modified by the mail server as
well, and that is why SA's SPF check fails!
Why would it do that ? I will read up on that.


because, after forwarding is done, SPF would fail - that is why SRS applied.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Chernobyl was an Windows 95 beta test site.

Re: SPF Fail for Amazon mails, although mail headers say its a pass

[MarcelM]

Ahh... I see. So probably other headers are modified by the mail server as
well, and that is why SA's SPF check fails! 
Why would it do that ? I will read up on that.
Thank you!



--
Sent from: http://spamassassin.1065346.n5.nabble.com/SpamAssassin-Users-f3.html

Re: SPF Fail for Amazon mails, although mail headers say its a pass

[Matus UHLAR - fantomas]

On 05.06.19 23:06, MarcelM wrote:

These are the full headers. (Sorry, did not realise all emails get redacted)

https://pastebin.com/Z6hkL9hD


the mails didn't get redacted. The headers are quoted even on the pastebin
example.


and the non-forwared mail header too:

https://pastebin.com/WGM0aYrh


this one looks better.


Does not look like SRS (but a good readup, something learned again).
I really don't get this. The spf record for 'amazon.de' states : v=spf1
include:amazon.com -all
So I understand the SPF fail, since there is no 'amazon.com' to be found
here... but the mail server obviously thinks it's valid, which it probably
is, at least it is legit and amazon should have correctly configured mail
servers - I hope...


Return-Path: 


this is mail from ampel-24.de, not amazon.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
"Where do you want to go to die?" [Microsoft]

Re: SPF Fail for Amazon mails, although mail headers say its a pass

[MarcelM]

These are the full headers. (Sorry, did not realise all emails get redacted)

https://pastebin.com/Z6hkL9hD

and the non-forwared mail header too:

https://pastebin.com/WGM0aYrh

Does not look like SRS (but a good readup, something learned again).
I really don't get this. The spf record for 'amazon.de' states : v=spf1
include:amazon.com -all
So I understand the SPF fail, since there is no 'amazon.com' to be found
here... but the mail server obviously thinks it's valid, which it probably
is, at least it is legit and amazon should have correctly configured mail
servers - I hope...



--
Sent from: http://spamassassin.1065346.n5.nabble.com/SpamAssassin-Users-f3.html

Re: SPF Fail for Amazon mails, although mail headers say its a pass

[MarcelM]

Thanks Bill, I will check that!



--
Sent from: http://spamassassin.1065346.n5.nabble.com/SpamAssassin-Users-f3.html

Re: SPF Fail for Amazon mails, although mail headers say its a pass

[Bill Cole]

On 5 Jun 2019, at 10:09, MarcelM wrote:

I am not sure how Spamassassin checks SPF, but this mail did pass and 
fail at

the same time!Spamassassin scored a fail, although it passed...

This is the complete header of an example:



[hopelessly mangled header block snipped]

Please, if you can't include pristine plain text headers in a message 
with the original formatting intact (which is apparently impossible when 
posting via Nabble) use Pastebin or a similar facility and include the 
link.


Either SA is checking SPF itself, and doing it wrong, or SA is 
checking for

the Received-SPF header, but not detecting it. Or not ?


SA will only trust a Received-SPF that it can be certain was added by a 
trusted relay, so if you don't have trusted_networks and 
internal_networks set up correctly, SA won't use the header. In this 
case it looks like you're doing SRS, and if SA is given the message 
after SRS has been done, it could be testing the rewritten address 
against what it sees as the last external relay point. I don't believe 
it is possible for SA to figure out what SRS has done to a message and 
check the validity of a relay that occurred before the SRS was done.


To figure out what exactly SA is doing, test a message using the "-D 
all" option and look for the analysis of Received and Received-SPF 
headers. Also try reading the details of how SA does SPF: 'perldoc 
Mail::SpamAssassin::Plugin::SPF'


--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

Re: SPF

[Bill Cole]

On 6 May 2019, at 17:10, Grant Taylor wrote:

> On 5/3/19 2:02 PM, Bill Cole wrote:
>> If the signer domain and the From header domain match, a valid DKIM 
>> signature that includes the From header is authentication of the From header 
>> to the limits of DNS trustworthiness and trust in the integrity of the 
>> domain's authority.
>
> Which section of RFC 6376 supports this statement?

The parts that use the word "domain."

There is a basic premise grounded in the the definition of domain names and 
buttressed by the use of domain names in effectively everything else that 
domains have a unitary executive: that the entity which publishes a public key 
in a DKIM record, the entity that signs mail with the corresponding private 
key, and the entity that controls the email local-part namespace for the domain 
are all one entity, as far as the world is concerned.

> I just re-read large chunks of RFC 6376 and see verbiage that states the 
> opposite.  All of which seem to me to specifically avoid drawing any 
> conclusion about the authorship of a message within the context of DKIM.  
> Further, such conclusions are left to other things making policy decisions 
> based on DKIM results.
>
> | § 3.11 - Relationship between SDID and AUID
> |
> | INFORMATIVE DISCUSSION: This document does not require the value
> | of the SDID or AUID to match an identifier in any other message
> | header field.
>
> DKIM does not require SDID or AUID to match any other header field.  As such, 
> DKIM itself can't be relied upon as authentication of other header fields.

Non sequitur.

DKIM itself does not require any sort of match. It is entirely valid for a 
signer to sign with a key whose domain is unrelated to any domain in the 
message or its envelope.

However, in ALL cases the DKIM signer is claiming responsibility for the 
message being signed. What that claim is worth in all cases is not specified by 
DKIM. In a special case (DKIM signer domain = From address domain) the 
conceptual nature of the DNS implies that the claim of responsibility is de 
facto authentication.


> | This requirement is, instead, an Assessor policy issue.
>
> Per § 2.7, the Identity Assessor "consumed DKIM's payload" which tells me 
> that it is not part of DKIM.  I believe that "Other DKIM (and non-DKIM) 
> values can also be used by the Identity Assessor…."  supports the fact that 
> the Identity Assessor is external to DKIM.

Yes.
This does not mean that it does not exist or that it doesn't need to follow 
some basic rules.
SpamAssassin has a DKIM_VALID_AU rule because a basic rule it understands about 
assessing identity is that a domain signer should know whether a From in its 
domain is valid.



-- 
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole


signature.asc
Description: OpenPGP digital signature

Re: SPF

[RW]

On Mon, 6 May 2019 15:10:07 -0600
Grant Taylor wrote:


> Many university IT departments claim responsibility for what the 
> university's staff and students do while decidedly NOT authenticating 
> anyone.  

DKIM author domain signing provides strong authentication where it
matters, which is where the domain owner cares. For well-known
addresses it's merely a matter of putting them on a separate domain or
sub-domain from user accounts. 

If a university server accepts spoofed submissions then they clearly
don't care that one user can spoof another.

Re: SPF

[Grant Taylor]

On 5/3/19 11:41 PM, Bill Cole wrote:
This is all true of any authentication mechanism: if control of 
authenticating credentials is lost, the authentication is worthless.


Agreed.

For example, if someone can control the DNS for tnetconsulting.net, 
they can very likely get Comodo to reissue your S/MIME cert and send 
it to them instead of you. At that point they can sign mail as you in a 
manner that normally would be seen as more robust and more reliable than 
DKIM.


Yep.

Hence the repudiation comment.

I'm afraid that our industry is painting themselves into a corner and we 
may not realize it until it's too late.


"Jury of peers" is going to be more and more problematic.  Not only will 
it need to be peers that understand the technologies in play, but peers 
that understand the ramifications of the weird failure modes.


BUT: you have an advantage over many victims of DNS compromise in that 
tnetconsulting.net has implemented DNSSEC.


}:-)

As long as they only compromise my DNS and /don't/ compromise my 
registrar, things should be relatively okay.  At least in that there is 
a higher authority that can be used to correct things top down.


My understanding is that DNSpionage & Sea Turtle did compromise things 
at the registrar level.  As such, they can alter the DS records and even 
overcome DNSSEC.


Also, DKIM is potentially less vulnerable to DNS compromise than 
PKI-reliant X.509 certificates (which DNSpionage & Sea Turtle target) 
because the protocol is designed to support short-lived keys distributed 
over DNSSEC so that a compromise of DNS needs to be persistent and 
stealthy to live longer than a signed TXT record.


Sure.  Smaller exposure window.



--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: SPF

[Grant Taylor]

On 5/3/19 2:02 PM, Bill Cole wrote:
If the signer domain and the From header domain match, a valid DKIM 
signature that includes the From header is authentication of the From 
header to the limits of DNS trustworthiness and trust in the integrity 
of the domain's authority. 


Which section of RFC 6376 supports this statement?

I just re-read large chunks of RFC 6376 and see verbiage that states the 
opposite.  All of which seem to me to specifically avoid drawing any 
conclusion about the authorship of a message within the context of DKIM. 
 Further, such conclusions are left to other things making policy 
decisions based on DKIM results.


| § 3.11 - Relationship between SDID and AUID
|
| INFORMATIVE DISCUSSION: This document does not require the value
| of the SDID or AUID to match an identifier in any other message
| header field.

DKIM does not require SDID or AUID to match any other header field.  As 
such, DKIM itself can't be relied upon as authentication of other header 
fields.


| This requirement is, instead, an Assessor policy issue.

Per § 2.7, the Identity Assessor "consumed DKIM's payload" which tells 
me that it is not part of DKIM.  I believe that "Other DKIM (and 
non-DKIM) values can also be used by the Identity Assessor…."  supports 
the fact that the Identity Assessor is external to DKIM.


| The purpose of such a linkage would be to authenticate the value in
| that other header field.  This, in turn, is the basis for applying a
| trust assessment based on the identifier value.

Sounds like the job of the Identity Assessor to me.

| Trust is a broad and complex topic, and trust mechanisms are subject
| to highly creative attacks.  The real-world efficacy of any but the
| most basic bindings between the SDID or AUID and other identities is
| not well established, nor is its vulnerability to subversion by an
| attacker.

Sounds like the DKIM spec is specifically trying to avoid a tricky issue 
and instead sticking to the simpler verifiable facts of has the message 
been modified since it was signed.


| Hence, reliance on the use of such bindings should be strictly
| limited.  In particular, it is not at all clear to what extent a
| typical end-user recipient can rely on any assurances that might be
| made by successful use of the SDID or AUID.

Sound to me like RFC 6376 is not even willing to loosely assert any 
relationship between the SDID / AUID / other headers.


To me, RFC 6376 goes out of it's way to avoid that conclusion and avoids 
asserting any authentication of the From: header.


This is authentication analogous to the authentication of the envelope 
sender provided by SPF.


I can't even agree to that.

I agree that SPF authenticates that a server is authorized to send an 
email purporting to be from an address in a domain that it's authorized 
to send email on behalf of.  But that does not extend to anything about 
the sending address, much less actual author of the message.


Email has an implicit trust that domains have a unitary executive, 
so we assume that a signer for a domain is not spoofable.


I concede that a server (SPF) / signer (DKIM) is authorized to send 
email on behalf of a domain.


I don't agree that the trust in the server extends into any statements 
about the authorship of a message.


Well, I know that the entity in control of DKIM signing for 
tnetconsulting.net was willing to claim responsibility for the message 
to which I am responding.
Willingness to accept responsibility for a message does not directly 
translate to vouching for it's authorship.


If the entity in control of DKIM signing for tnetconsulting.net is not 
an authoritative judge of "authenticity" (whatever that means... )


The "whatever that means" is EXTREMELY germane in this context.

for mail claiming to be from gtay...@tnetconsulting.net at the signing 
point, there is no way for me to detect that.


You have no way to know what, if any, authentication was done by the 
signing server.


As such, I believe the only safe thing that can be done is to assume 
that no authentication was done.


If the entity in control of DKIM signing for tnetconsulting.net chooses 
to sign mail claiming to be from an address in some other domain, there 
is no discernible reason to deem that to be authentication,


I believe this should be the behavior for all DKIM signature verification.

As outlined above, there is no guarantee that the SDID / AUID / From (et 
al.) headers correlate, much less actually authenticate.


RFC 6376 states the following in § 8.3.

| A more secure architecture involves sending messages through an
| outgoing MTA that can authenticate the submitter using existing
| techniques (e.g., SMTP Authentication),

Submitter ≠ Author

| possibly validate the message itself (e.g., verify that the header is
| legitimate and that the content passes a spam content check),

"legitimate" is rather nebulous.  Is it as simple as making sure that it 
is an email address?  Any email 

Re: SPF

[David Jones]

On 5/3/19 6:26 PM, Grant Taylor wrote:
> On 5/3/19 5:10 PM, Kevin A. McGrail wrote:
>> I guess if you lose control of your keys and/or your DNS is 
>> compromised, then yes, you have a DKIM issue.
> 
> This brings up a non-repudiation issue introduced by DKIM.
> 

This is similar to saying not to use HTTPS because there's a possibility 
that the web server's certificate private keys and DNS might be 
compromised so I am safer with plain HTTP.  If the DKIM signature aligns 
with the From: header domain and is valid, DKIM_VALID_AU means you know 
the source of the email and that is was not modified.  If the DKIM 
signature is any other domain, then it doesn't mean much from a spam/ham 
perspecitive which is why DKIM_VALID has a tiny default value.

> How can you successfully refute a DKIM-Signature if someone has your 
> signing keys.
> 

DKIM is only one of dozens of tools in the spam-fighting tool bag.  I 
use it in very specific cases when DKIM_VALID_AU rules hit or I use it 
in meta rules so that other conditions have to also be true.


> My quick skim of parts of RFC 6376 makes me think that it is dangerous 
> and discouraged to associate authentication based on DKIM-Signature, 
> even when the d= SDID (?) matches the From: header.
> 

I have been using SPF, DKIM, and DMARC for years now in specific meta 
rules based on patterns noticed in the mail flow through smtp.ena.net 
and it has worked pretty well.

1. Carefully whitelist_auth primary domains with user/human mailboxes 
since they can be compromised.  Some cases are worth the risk for 
difficult partners like legal, HR, financial, etc. with content that 
looks spammy and can hit high Bayes scores.

2. Whitelist_auth subdomains of primary domains are usually safe for 
systems-generated emails and mailing lists (whitelist_auth *.apache.org)


> Yet even more reason to reread RFC 6376 before replying to Bill's email.
> 
> Presently I'm comfortable in thinking that DKIM-Signature validation 
> meaning that the message has not changed in transit.  I'm not (yet) 
> comfortable drawing any conclusions about authentication.
> 

Analyze your email based on DKIM_VALID_AU hits and look for patterns. 
Based on your definition of spam vs UCE vs ham.  If there is enough 
volume, you should see how DKIM_VALID_AU and DMARC can enhance/extend 
SPF accuracy which was your original question.

-- 
David Jones

Re: SPF

[Kevin A. McGrail]

On 5/4/2019 12:48 AM, Grant Taylor wrote:
> The point being there are reasonable circumstances that someone else
> can DKIM sign messages as a victim.

Sure, your entire server can be compromised and there might be a mole in
the ministry.

Your premise started out with the From Header versus Envelope for SPF
and now has moved into DKIM being insecure if you don't maintain control
of your PKI security.

What is your point?

-- 
Kevin A. McGrail
Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Re: SPF

[Bill Cole]

On 4 May 2019, at 0:48, Grant Taylor wrote:

> On 5/3/19 5:51 PM, Kevin A. McGrail wrote:
>> If your key is compromised, generate another and publish it on DNS.
>
> That requires knowing that the key is compromised.
>
> It really helps to know that an APT is going on to know that your key has 
> been compromised.
>
> The point being there are reasonable circumstances that someone else can DKIM 
> sign messages as a victim.

This is all true of any authentication mechanism: if control of authenticating 
credentials is lost, the authentication is worthless.

For example, if someone can control the DNS for tnetconsulting.net, they can 
very likely get Comodo to reissue your S/MIME cert and send it to them instead 
of you. At that point they can sign mail as you in a manner that normally would 
be seen as more robust and more reliable than DKIM. BUT: you have an advantage 
over many victims of DNS compromise in that tnetconsulting.net has implemented 
DNSSEC.

Also, DKIM is potentially less vulnerable to DNS compromise than PKI-reliant 
X.509 certificates (which DNSpionage & Sea Turtle target) because the protocol 
is designed to support short-lived keys distributed over DNSSEC so that a 
compromise of DNS needs to be persistent and stealthy to live longer than a 
signed TXT record.

signature.asc
Description: OpenPGP digital signature

Re: SPF

[Grant Taylor]

On 5/3/19 5:51 PM, Kevin A. McGrail wrote:

If your key is compromised, generate another and publish it on DNS.


That requires knowing that the key is compromised.

It really helps to know that an APT is going on to know that your key 
has been compromised.


The point being there are reasonable circumstances that someone else can 
DKIM sign messages as a victim.




--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: SPF

[Kevin A. McGrail]

If your key is compromised, generate another and publish it on DNS.

On Fri, May 3, 2019, 19:43 Grant Taylor  wrote:

> On 5/3/19 5:10 PM, Kevin A. McGrail wrote:
> > I guess if you lose control of your keys and/or your DNS is compromised,
> > then yes, you have a DKIM issue.
>
> This brings up a non-repudiation issue introduced by DKIM.
>
> How can you successfully refute a DKIM-Signature if someone has your
> signing keys.
>
> My quick skim of parts of RFC 6376 makes me think that it is dangerous
> and discouraged to associate authentication based on DKIM-Signature,
> even when the d= SDID (?) matches the From: header.
>
> Yet even more reason to reread RFC 6376 before replying to Bill's email.
>
> Presently I'm comfortable in thinking that DKIM-Signature validation
> meaning that the message has not changed in transit.  I'm not (yet)
> comfortable drawing any conclusions about authentication.
>
>
>
> --
> Grant. . . .
> unix || die
>
>

Re: SPF

[Grant Taylor]

On 5/3/19 5:10 PM, Kevin A. McGrail wrote:
I guess if you lose control of your keys and/or your DNS is compromised, 
then yes, you have a DKIM issue.


This brings up a non-repudiation issue introduced by DKIM.

How can you successfully refute a DKIM-Signature if someone has your 
signing keys.


My quick skim of parts of RFC 6376 makes me think that it is dangerous 
and discouraged to associate authentication based on DKIM-Signature, 
even when the d= SDID (?) matches the From: header.


Yet even more reason to reread RFC 6376 before replying to Bill's email.

Presently I'm comfortable in thinking that DKIM-Signature validation 
meaning that the message has not changed in transit.  I'm not (yet) 
comfortable drawing any conclusions about authentication.




--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: SPF

[Grant Taylor]

On 5/3/19 4:47 PM, Kevin A. McGrail wrote:
Unless you have the private key matching the public key in 
DNS of a domain, that's the benefit of a DKIM signature.


I was referring to exactly that.

As in the real ena.com being compromised and attackers taking a copy of 
their private key.


See recent reports of DNSpionage and Sea Turtle.  (Cisco Talos)

There's also the fact that DNSpionage attackers can modify the contents 
of DNS to fit their needs.




--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: SPF

[Kevin A. McGrail]

On 5/3/2019 6:53 PM, Grant Taylor wrote:
> On 5/3/19 4:47 PM, Kevin A. McGrail wrote:
>> Unless you have the private key matching the public key in DNS of a
>> domain, that's the benefit of a DKIM signature.
>
> I was referring to exactly that.
>
I guess if you lose control of your keys and/or your DNS is compromised,
then yes, you have a DKIM issue.

Re: SPF

[Kevin A. McGrail]

On 5/3/2019 6:40 PM, Grant Taylor wrote:
> I think that I could sign as d=ena.com if I had access to their keys.
> Which obviously I / my server should not.
>
> I need to reread if there is any protection in DKIM to detect such
> malicious use of the spoofed domain's keys.  My current understanding
> is that there is not any such protection in DKIM.

Unless you have the private key matching the public key in DNS of a
domain, that's the benefit of a DKIM signature.  You might want to read
more about asymmetric crypto:
https://en.wikipedia.org/wiki/Public-key_cryptography

Regards,

KAM


-- 
Kevin A. McGrail
Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Re: SPF

[Grant Taylor]

On 5/3/19 4:35 PM, RW wrote:
But if you sign it with d=ena.com it wont pass as valid, unless you have 
also gained control of the DNS for ena.com.


I was referring to signing it with d=tnetconsulting.net.

I need to reread RFC 6376 to comment further.  But at this point, I 
think that I could sign as d=ena.com if I had access to their keys. 
Which obviously I / my server should not.


I need to reread if there is any protection in DKIM to detect such 
malicious use of the spoofed domain's keys.  My current understanding is 
that there is not any such protection in DKIM.




--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: SPF

[RW]

On Fri, 3 May 2019 13:28:00 -0600
Grant Taylor wrote:

> On 5/3/19 11:53 AM, David Jones wrote:
> > Not completely true as long as domain/DNS control is not
> > compromised.  
> 
> How is it not completely true?
> 
> My server can apply a DKIM signature to an outgoing email with a
> From: header of djo...@ena.com.

But if you sign it with d=ena.com it wont pass as valid, unless you
have also gained control of the DNS for ena.com.

Re: SPF

[Bill Cole]

On 3 May 2019, at 12:30, Grant Taylor wrote:

> On 5/3/19 9:48 AM, Bill Cole wrote:
>> An entirely different mechanism (DKIM) exists to verify From headers.
>
> DKIM is only positive confirmation that the (signed) headers (and body 
> content) has not changed since the signature was applied.

RFC6376:

  1.  Introduction

 DomainKeys Identified Mail (DKIM) permits a person, role, or
 organization to claim some responsibility for a message by
 associating a domain name [RFC1034] with the message [RFC5322], which
 they are authorized to use.

If the signer domain and the From header domain match, a valid DKIM signature 
that includes the From header is authentication of the From header to the 
limits of DNS trustworthiness and trust in the integrity of the domain's 
authority. This is authentication analogous to the authentication of the 
envelope sender provided by SPF. Email has an implicit trust that domains have 
a unitary executive, so we assume that a signer for a domain is not spoofable.


> DKIM does nothing to verify the authenticity of what was signed (at the time 
> it was signed).

Well, I know that the entity in control of DKIM signing for tnetconsulting.net 
was willing to claim responsibility for the message to which I am responding 
which claims to be from gtay...@tnetconsulting.net which was substantially 
unchanged between the signing point and my mail server. If the entity in 
control of DKIM signing for tnetconsulting.net is not an authoritative judge of 
"authenticity" (whatever that means... ) for mail claiming to be from 
gtay...@tnetconsulting.net at the signing point, there is no way for me to 
detect that.

If the entity in control of DKIM signing for tnetconsulting.net chooses to sign 
mail claiming to be from an address in some other domain, there is no 
discernible reason to deem that to be authentication, although it is still 
(according to the RFC) a claim of responsibility for that mail by the entity in 
control of DKIM signing for tnetconsulting.net.

So as Dave said: DKIM_VALID_AU has authentication value where DKIM_VALID alone 
has none unless you have some reason to trust the signer.

> ARC (not DMARC) is a similar signature of what comes in to detect 
> modification down stream.

DMARC is not a signature. It is a statement of policy recommendations for how 
to assess and handle messages which purport to be from a domain but either fail 
SPF checking or DKIM validation *and alignment with From*

ARC is a totally different thing, authenticating chain-of-custody.

-- 
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole


signature.asc
Description: OpenPGP digital signature

Re: SPF

[Grant Taylor]

On 5/3/19 11:53 AM, David Jones wrote:

Not completely true as long as domain/DNS control is not compromised.


How is it not completely true?

My server can apply a DKIM signature to an outgoing email with a From: 
header of djo...@ena.com.


Nothing about my server's DKIM signature verifies the contents of the 
From: header is accurate.


Technically this is correct but the fact that it's signed and matches 
the author's domain in the From: header provides authenticity of the 
origin of the email.  In SA rules this would be DKIM_VALID_AU hits.


No it does not.

First, you have introduced an additional requirement that the signature 
matches the author's domain.  This is not a requirement of DKIM.


Second, it's (hypothetically) possible for me to get an account on the 
ena.com email server and send email as djones@ with a signature matching 
the ena.com domain.  But such message is decidedly not from you.  Ergo, 
DKIM does nothing to verify the authenticity of what was signed.


For example, Microsoft signs customer emails using the tenant's subdomain 
under onmicrosoft.com.  All this confirms is the email came from the 
Office 365 platform with the original content unmodified.


It only confirms that the content is unmodified after the DKIM signature 
is applied.  DKIM does nothing for any part of email before the DKIM 
signature is applied.


Microsoft could modify the message before they apply the signature.

Since it doesn't align with the From: domain, DKIM really means nothing 
from a forged/spoofed (negative) perspective.  DKIM can prove the positive 
that it was not forged/spoofed when it aligns and hits DKIM_VALID_AU.


See above about signatures matching domains.

I really don't think there is any part of the DKIM specification that 
matters if the signing domain matches the purported sending domain. 
There are other things that deduce things from that information.  But to 
the best of my knowledge, they are outside of the DKIM specification. 
Please point to something for me to read if you think I'm wrong.


I am not completely clear on ARC but I though it's objective is to provide 
a "chain of custody" as email goes through mail servers so receiving mail 
servers can authenticate the origin.  I was thinking it's something like 
a combination of SPF (validation) and DKIM (authentication).


My understanding is that ARC primarily provides a way for a server to 
state "This is who I received the message from, with these parameters." 
Then if down stream servers trust the ARC signer, they can apply their 
own (SPF) processing using the asserted information from the trusted ARC 
signer.




--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: SPF

[David Jones]

On 5/3/19 11:30 AM, Grant Taylor wrote:
> On 5/3/19 9:48 AM, Bill Cole wrote:
>> An entirely different mechanism (DKIM) exists to verify From headers.
> 
> DKIM is only positive confirmation that the (signed) headers (and body 
> content) has not changed since the signature was applied.
> 

Not completely true as long as domain/DNS control is not compromised.

> DKIM does nothing to verify the authenticity of what was signed (at the 
> time it was signed).
> 

Technically this is correct but the fact that it's signed and matches 
the author's domain in the From: header provides authenticity of the 
origin of the email.  In SA rules this would be DKIM_VALID_AU hits.

For example, Microsoft signs customer emails using the tenant's 
subdomain under onmicrosoft.com.  All this confirms is the email came 
from the Office 365 platform with the original content unmodified. 
Since it doesn't align with the From: domain, DKIM really means nothing 
from a forged/spoofed (negative) perspective.  DKIM can prove the 
positive that it was not forged/spoofed when it aligns and hits 
DKIM_VALID_AU.

> ARC (not DMARC) is a similar signature of what comes in to detect 
> modification down stream.
> 

I am not completely clear on ARC but I though it's objective is to 
provide a "chain of custody" as email goes through mail servers so 
receiving mail servers can authenticate the origin.  I was thinking it's 
something like a combination of SPF (validation) and DKIM (authentication).

-- 
David Jones

Re: SPF

[Grant Taylor]

On 5/3/19 9:48 AM, Bill Cole wrote:

An entirely different mechanism (DKIM) exists to verify From headers.


DKIM is only positive confirmation that the (signed) headers (and body 
content) has not changed since the signature was applied.


DKIM does nothing to verify the authenticity of what was signed (at the 
time it was signed).


ARC (not DMARC) is a similar signature of what comes in to detect 
modification down stream.




--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: SPF

[Daniele Duca]

Take your email in example:

envelope from: 
users-return-120376-duca=staff.spin...@spamassassin.apache.org

body from:  maj...@gmail.com

SPF for gmail.com: v=spf1 redirect=_spf.google.com

You see that in case of mailing lists (and ESPs and possibly every other 
VERP case) a check on the body from would fail even if the original 
sender intended to send that email.


Daniele

On 03/05/19 17:24, user321 wrote:

But I have a feeling this would be extremely effective in dealing with
spoofed emails. They are often having borderline score around the blocking
point, so that kind of rule with relatively low score could be the last
straw on the camel's back, don't you agree?

cheers
user321



--
Sent from: http://spamassassin.1065346.n5.nabble.com/SpamAssassin-Users-f3.html

Re: SPF

[David Jones]

On 5/3/19 9:47 AM, RW wrote:
> On Fri, 3 May 2019 06:55:40 -0700 (MST)
> user321 wrote:
> 
>> Any reason why SA is checking for SPF against envelope from not the
>> header from?
> 

See the SPF link on this page:

https://blog.returnpath.com/how-to-explain-dmarc-in-plain-english/

> Because that's how SPF works.
> 
>> I am rejecting the SPF_FAIL e-mails on Postfix (-all only), but still
>> spammers can forge the header from field.
>> Can I change SPF plugin to work with header from? If yes how?
>> What are the pros and cons of that?
> 
> Generally you don't want to enforcing a modified standard that no one
> else knows about.
> 
> DMARC solves the problem by requiring that for a DMARC pass from SPF
> the envelope address has to be aligned with the from header address.
> 

FYI,

If opendmarc is setup in your MTA with local SA rules...

DMARC_PASS = SPF_PASS and From: domain aligns with envelope-domain
  _OR_  DKIM_VALID_AU (DKIM_SIGNED and aligns with the From: domain)

I am rejecting DMARC failures of incoming domains with p=reject within 
opendmarc then have these rules in SA:

(change the rule below to match your header in opendmarc.conf)

headerDMARC_PASS Authentication-Results =~ /smtp\.ena\.net; dmarc=pass/
describe  DMARC_PASS DMARC check passed
score DMARC_PASS 0.001

headerDMARC_FAIL Authentication-Results =~ /smtp\.ena\.net; dmarc=fail/
describe  DMARC_FAIL DMARC check failed
score DMARC_FAIL 0.001

headerDMARC_NONE Authentication-Results =~ /smtp\.ena\.net; dmarc=none/
describe  DMARC_NONE DMARC check neutral
score DMARC_NONE 0.001

-- 
David Jones

Re: SPF

[user321]

But I have a feeling this would be extremely effective in dealing with
spoofed emails. They are often having borderline score around the blocking
point, so that kind of rule with relatively low score could be the last
straw on the camel's back, don't you agree?

cheers
user321



--
Sent from: http://spamassassin.1065346.n5.nabble.com/SpamAssassin-Users-f3.html

Re: SPF

[RW]

On Fri, 3 May 2019 06:55:40 -0700 (MST)
user321 wrote:

> Any reason why SA is checking for SPF against envelope from not the
> header from?

Because that's how SPF works.

> I am rejecting the SPF_FAIL e-mails on Postfix (-all only), but still
> spammers can forge the header from field.
> Can I change SPF plugin to work with header from? If yes how?
> What are the pros and cons of that?

Generally you don't want to enforcing a modified standard that no one
else knows about. 

DMARC solves the problem by requiring that for a DMARC pass from SPF
the envelope address has to be aligned with the from header address.

Re: SPF

[Christian Grunfeld]

El vie., 3 may. 2019 a las 11:13, user321 () escribió:

> Any reason why SA is checking for SPF against envelope from not the header
> from?
>
> This is what SPF is made for

>
> cheers
> user
>
>
>
> --
> Sent from:
> http://spamassassin.1065346.n5.nabble.com/SpamAssassin-Users-f3.html
>

Re: SPF weirdness...

[Grant Taylor]

On 1/15/19 8:02 PM, David B Funk wrote:
It's a bit tricky to implement a milter correctly because people often 
don't understand that the message which sendmail hands to a milter is 
as-received from the incoming network connection.


Any locally added stuff (EG the "Received:" header) isn't in that 
milter stream.


Thus the milter must completely/correctly synthesize all locally added 
headers.


I might have known that at some point in the past but I definitely 
forgot about it until today.


Actually the spamass-milter method (calling spamc) makes it easier 
to debug.


Wait.  What‽

I thought spamass-milter used the same protocol to communicate with 
spamd that spamc does.  Somehow I was not aware that spamass-milter 
actually used spamc to communicate with spamd.


Just create a script which wraps spamc in-between a couple of "tee"s to 
capture stdin & stdout and you'll have everything you want to know. 





A simple example which ignores signal handling:

  #!/bin/sh
  # 'spamc' debugging script
  FILE_NAME="/var/tmp/spamc-transcript-$$"
  echo "spamc args: $*" "" > ${FILE_NAME}.in
  tee -a ${FILE_NAME}.in | /real/path/to/spamc "$@" | tee ${FILE_NAME}.out

Adjust paths as needed.


*nod*nod*

Thank you for the information Dave.



--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: SPF weirdness...

[David B Funk]

On Tue, 15 Jan 2019, Bill Cole wrote:


On 15 Jan 2019, at 15:05, Grant Taylor wrote:


I will investigate to see if spamass-milter can fabricate a satisfactory 
Received: header.


A quick look at the issue tracker for it implies that it does so. A milter that 
actually works with SA really needs to.

Unfortunately, it is a nuisance to debug spamass-milter because it talks to 
spamc which talks to spamd, so you need to give debug flags to the 
spamass-milter process and spamd to see exactly what's going on.


This is a very real question.
It's a bit tricky to implement a milter correctly because people often don't 
understand that the message which sendmail hands to a milter is as-received 
from the incoming network connection.

Any locally added stuff (EG the "Received:" header) isn't in that milter stream.
Thus the milter must completely/correctly synthesize all locally added headers.

Actually the spamass-milter method (calling spamc) makes it easier to debug.
Just create a script which wraps spamc in-between a couple of "tee"s to capture 
stdin & stdout and you'll have everything you want to know.


A simple example which ignores signal handling:

 #!/bin/sh
 # 'spamc' debugging script
 FILE_NAME="/var/tmp/spamc-transcript-$$"
 echo "spamc args: $*" "" > ${FILE_NAME}.in
 tee -a ${FILE_NAME}.in | /real/path/to/spamc "$@" | tee ${FILE_NAME}.out

Adjust paths as needed.


--
Dave Funk  University of Iowa
College of Engineering
319/335-5751   FAX: 319/384-0549   1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include 
Better is not better, 'standard' is better. B{

Re: SPF weirdness...

[Bill Cole]

On 15 Jan 2019, at 15:05, Grant Taylor wrote:

> I will investigate to see if spamass-milter can fabricate a satisfactory 
> Received: header.

A quick look at the issue tracker for it implies that it does so. A milter that 
actually works with SA really needs to.

Unfortunately, it is a nuisance to debug spamass-milter because it talks to 
spamc which talks to spamd, so you need to give debug flags to the 
spamass-milter process and spamd to see exactly what's going on.

signature.asc
Description: OpenPGP digital signature

Re: SPF weirdness...

[Grant Taylor]

On 01/15/2019 12:59 PM, Bill Cole wrote:
There are at many different milters that can use SpamAssassin listed at 
https://wiki.apache.org/spamassassin/IntegratedInMta#Integrated_into_Sendmail. 
Some links there may be dead.


I am using spamass-milter, and spfmilter, both connected to Sendmail.


SpamAssassin is not a milter.


Agreed.  Sorry for the poor choice of words.

SpamAssassin knows nothing about message parameters passed through the 
milter interface between a MTA and a milter. The ONLY message data that 
SpamAssassin knows about is what it gets in a RFC822/2822/5322 format 
message with parseable headers.


I was not consciously aware of that or the implications there of.

A milter that uses SpamAssassin can modify the message that it 
receives via the milter interface before passing it to SpamAssassin for 
analysis. This allows the milter to inform SpamAssassin of facts that 
SpamAssassin can use, such as the SMTP client address, envelope sender 
and recipients, and whatever else it gets from the MTA. For SpamAssassin 
to do SPF calculations it needs to have a Received header and envelope 
sender, which can be embedded in headers that are added by a milter that 
uses SpamAssassin.


I will investigate to see if spamass-milter can fabricate a satisfactory 
Received: header.


Thank you for the pointers.



--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: SPF weirdness...

[Bill Cole]

On 15 Jan 2019, at 14:24, Grant Taylor wrote:

> On 01/15/2019 11:39 AM, Bill Cole wrote:
>> This strikes me as a flaw in whatever milter you're using. Some (e.g. 
>> MIMEDefang) milters deal with the fact that they don't get a local Received 
>> header by constructing one from what they know before passing the message to 
>> SA.
>
> The SPF milter is constructing the header.  I assume that it's doing so 
> properly.  At least the headers I see coming out of the MTA are correct.
>
> I think that SpamAssassin is looking for a header that isn't there yet. -  
> Both SpamAssassin and my SPF filter are hooked into the same MTA as milters.  
> So both of them see the message before it's accepted and all headers new are 
> added.
>
> I don't know if the SPF milter can add the header sooner, or if that is 
> controlled by the MTA.
>
> I would also like SpamAssassin to use the information available to it via the 
> milter interface instead of relying on a header.

Let me clarify...

There are at many different milters that can use SpamAssassin listed at 
https://wiki.apache.org/spamassassin/IntegratedInMta#Integrated_into_Sendmail. 
Some links there may be dead.

SpamAssassin is not a milter. SpamAssassin knows nothing about message 
parameters passed through the milter interface between a MTA and a milter. The 
ONLY message data that SpamAssassin knows about is what it gets in a 
RFC822/2822/5322 format message with parseable headers.

A milter that uses SpamAssassin can modify the message that it receives via the 
milter interface before passing it to SpamAssassin for analysis. This allows 
the milter to inform SpamAssassin of facts that SpamAssassin can use, such as 
the SMTP client address, envelope sender and recipients, and whatever else it 
gets from the MTA. For SpamAssassin to do SPF calculations it needs to have a 
Received header and envelope sender, which can be embedded in headers that are 
added by a milter that uses SpamAssassin.

signature.asc
Description: OpenPGP digital signature

Re: SPF weirdness...

[Grant Taylor]

On 01/15/2019 11:39 AM, Bill Cole wrote:
This strikes me as a flaw in whatever milter you're using. Some 
(e.g. MIMEDefang) milters deal with the fact that they don't get a local 
Received header by constructing one from what they know before passing 
the message to SA.


The SPF milter is constructing the header.  I assume that it's doing so 
properly.  At least the headers I see coming out of the MTA are correct.


I think that SpamAssassin is looking for a header that isn't there yet. 
-  Both SpamAssassin and my SPF filter are hooked into the same MTA as 
milters.  So both of them see the message before it's accepted and all 
headers new are added.


I don't know if the SPF milter can add the header sooner, or if that is 
controlled by the MTA.


I would also like SpamAssassin to use the information available to it 
via the milter interface instead of relying on a header.




--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: SPF weirdness...

[Bill Cole]

On 15 Jan 2019, at 12:15, Grant Taylor wrote:

> On 01/15/2019 09:24 AM, Kevin A. McGrail wrote:
>> What is your glue for SA?  Is it getting the received header you are 
>> expecting in time for the parsing?
>
> Both SA and my spfmilter are are milters on the same inbound Internet edge 
> MTA.
>
> I will have to research to see if the header is added by the time that SA 
> checks things.
>
> I do know that the Received: header isn't there by the time that SA runs.  I 
> don't know if my MTA has added the proper Authentication-Results: header yet 
> or not.
>
> …
>
> As sure as I type that, "…the Received: header isn't there…", which may mean 
> that SA is running the contents of the previous Received: header through SPF 
> checks.
>
> …
>
> That seems to be part of the problem.
>
> Thank you Kevin.  I now have something more specific to investigate.

This strikes me as a flaw in whatever milter you're using. Some (e.g. 
MIMEDefang) milters deal with the fact that they don't get a local Received 
header by constructing one from what they know before passing the message to SA.


signature.asc
Description: OpenPGP digital signature

Re: SPF weirdness...

[Grant Taylor]

On 01/15/2019 09:24 AM, Kevin A. McGrail wrote:
What is your glue for SA?  Is it getting the received header you are 
expecting in time for the parsing?


Both SA and my spfmilter are are milters on the same inbound Internet 
edge MTA.


I will have to research to see if the header is added by the time that 
SA checks things.


I do know that the Received: header isn't there by the time that SA 
runs.  I don't know if my MTA has added the proper 
Authentication-Results: header yet or not.


…

As sure as I type that, "…the Received: header isn't there…", which may 
mean that SA is running the contents of the previous Received: header 
through SPF checks.


…

That seems to be part of the problem.

Thank you Kevin.  I now have something more specific to investigate.



--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: SPF weirdness...

[Grant Taylor]

On 01/15/2019 09:36 AM, Bill Cole wrote:
Check both the contents and documentation of trusted_networks, 
msa_networks, and internal_networks.


Will do.

If SA thinks a prior hop is through a machine that writes trustworthy 
Received headers and is a normal part of your relay path, it will check 
SPF there.


SA is running at the inbound Internet edge.  There are no other hosts 
that are part of my organization.


There MAY be a design bug there. I'm not sure how SA deals with a 
machine you trust and which is a normal inbound relay that also is 
SPF-approved for mail it gets from other places. Maybe msa_networks can 
solve this.


I don't think this is applicable when SA is at the inbound Internet edge.

I'll check the referenced documentation as soon as time permits.

Thank you Bill.



--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: SPF weirdness...

[Bill Cole]

On 15 Jan 2019, at 11:08, Grant Taylor wrote:

Does anybody know off the top of their head—don't dig, I'll do that 
later—what might cause SpamAssassin to apply SPF processing to 
earlier Received: headers (lower in the message source)?


Check both the contents and documentation of trusted_networks, 
msa_networks, and internal_networks.


I'm seeing SpamAssassin claim that a message failed SPF processing 
based on chronologically earlier internal Received: headers.  
Conversely, the connection to my SMTP server are perfectly acceptable 
with the published SPF record.


If SA thinks a prior hop is through a machine that writes trustworthy 
Received headers and is a normal part of your relay path, it will check 
SPF there.


There MAY be a design bug there. I'm not sure how SA deals with a 
machine you trust and which is a normal inbound relay that also is 
SPF-approved for mail it gets from other places. Maybe msa_networks can 
solve this.


I just noticed this and will look into it further later as soon as 
time permits.  I'm hoping that someone may have a 15 second knee-jerk 
"check this or that" type response.


Thank you in advance.



--
Grant. . . .
unix || die

Re: SPF check though external relay

[Sean Greenslade]

>On 11.11.17 20:06, Sean Greenslade wrote:
>>SPF checks the final server that transmits the mail. If you are using
>a relay server, that server will need to be in the SPF records.
>
>no. Only outgoing mail servers really need to be in SPF records.

Sorry, I misread the original message and thought this was in reference to 
outgoing mail.

--Sean

Re: SPF check though external relay

[Matus UHLAR - fantomas]

On November 11, 2017 5:31:08 PM PST, Stephan Herker  wrote:

I'm running spam assassin default configuration which checks spf
records.  In my case I received an email and it checked if the last
relay was a valid sender for SPF.  The last relay was a server I have
in
the cloud, so it failed SPF even though original sending server is on
senders SPF record.  Should I disable SPF checks or is there a
configuration change I need to make?


On 11.11.17 20:06, Sean Greenslade wrote:

SPF checks the final server that transmits the mail. If you are using a relay 
server, that server will need to be in the SPF records.


no. Only outgoing mail servers really need to be in SPF records.

for the incoming mail you need either to make sure your MX adds Received-SPF
headers properly (apparentlky they don't), or set
"ignore_received_spf_header 0" and set up trusted_networks and
internal_networks properly, so SA knows which header to use for SPF checks.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Emacs is a complicated operating system without good text editor.

Re: SPF check though external relay

[RW]

On Sat, 11 Nov 2017 17:31:08 -0800
Stephan Herker wrote:

> I'm running spam assassin default configuration which checks spf 
> records.  In my case I received an email and it checked if the last 
> relay was a valid sender for SPF.  The last relay was a server I have
> in the cloud, 

You probably need to setup internal_networks to include that server.

Re: SPF check though external relay

[David Jones]

On 11/11/2017 07:31 PM, Stephan Herker wrote:
I'm running spam assassin default configuration which checks spf 
records.  In my case I received an email and it checked if the last 
relay was a valid sender for SPF.  The last relay was a server I have in 
the cloud, so it failed SPF even though original sending server is on 
senders SPF record.  Should I disable SPF checks or is there a 
configuration change I need to make?


Thanks



If you have a relay server in the cloud that you manage/control/trust, 
it should be listed in your internal_networks and/or trusted_networks 
(probably both).  This will cause SA to look one hop back and should fix 
a number of network-based checks like RBL checks in addition to the SPF 
check.


https://wiki.apache.org/spamassassin/TrustPath

--
David Jones

Re: SPF check though external relay

[Sean Greenslade]

On November 11, 2017 5:31:08 PM PST, Stephan Herker  wrote:
>I'm running spam assassin default configuration which checks spf 
>records.  In my case I received an email and it checked if the last 
>relay was a valid sender for SPF.  The last relay was a server I have
>in 
>the cloud, so it failed SPF even though original sending server is on 
>senders SPF record.  Should I disable SPF checks or is there a 
>configuration change I need to make?

SPF checks the final server that transmits the mail. If you are using a relay 
server, that server will need to be in the SPF records.

--Sean

Re: spf

[Matus UHLAR - fantomas]

On 26.02.17 20:04, Gokan Atmaca wrote:

% locate -i spf.pm
/usr/lib/perl5/Net/DNS/RR/SPF.pm
/usr/share/perl5/Mail/SPF.pm   # this is the module.
/usr/share/perl5/Mail/SpamAssassin/Plugin/SPF.pm


Hello

This module path:

[root@mail ~]# locate -i spf.pm
/opt/zimbra/common/lib/perl5/Mail/SPF.pm
/opt/zimbra/common/lib/perl5/Mail/SpamAssassin/Plugin/SPF.pm
/opt/zimbra/common/lib/perl5/Net/DNS/RR/SPF.pm
/opt/zimbra/common/lib/policyd-2.1/cbp/modules/CheckSPF.pm


you still haven't answered my first two questions:


On Sun, Feb 26, 2017 at 3:34 PM, Matus UHLAR - fantomas
 wrote:

what is wrong with existing and provided SPF_PASS rule (and the others)?



did you load loadplugin Mail::SpamAssassin::Plugin::SPF ?



--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
On the other hand, you have different fingers. 

Re: spf

[Gokan Atmaca]

> % locate -i spf.pm
> /usr/lib/perl5/Net/DNS/RR/SPF.pm
> /usr/share/perl5/Mail/SPF.pm   # this is the module.
> /usr/share/perl5/Mail/SpamAssassin/Plugin/SPF.pm

Hello

This module path:

[root@mail ~]# locate -i spf.pm
/opt/zimbra/common/lib/perl5/Mail/SPF.pm
/opt/zimbra/common/lib/perl5/Mail/SpamAssassin/Plugin/SPF.pm
/opt/zimbra/common/lib/perl5/Net/DNS/RR/SPF.pm
/opt/zimbra/common/lib/policyd-2.1/cbp/modules/CheckSPF.pm

On Sun, Feb 26, 2017 at 3:34 PM, Matus UHLAR - fantomas
 wrote:
>>> Do you have Mail-SPF perl module installed?
>
>
> On 26.02.17 15:20, Gokan Atmaca wrote:
>>
>> i am using zimbra. The module is being installed.
>>
>> /opt/zimbra/data/spamassassin/rules/25_spf.cf -| SPF module
>
>
> that is not SPF module, nor a perl module. try searching it:
>
> % locate -i spf.pm
> /usr/lib/perl5/Net/DNS/RR/SPF.pm
> /usr/share/perl5/Mail/SPF.pm   # this is the module.
> /usr/share/perl5/Mail/SpamAssassin/Plugin/SPF.pm
>
> you also nees "loadplugin Mail::SpamAssassin::Plugin::SPF" uncommended in
> spamassassin's init.pre
>
>> On Sun, Feb 26, 2017 at 2:36 PM, Matus UHLAR - fantomas
>>  wrote:
>>>
>>> On 26.02.17 13:46, Gokan Atmaca wrote:


 I want to check SPF. But it does not work. Can you help me with this?

 config:
 header SPF_GECER eval:check_for_spf_pass()
 describe SPF_GECER   SPF: sender matches SPF record
 score SPF_GECER  -0.5
>>>
>>>
>>>
>>> what is wrong with existing and provided SPF_PASS rule (and the others)?
>>> did you load loadplugin Mail::SpamAssassin::Plugin::SPF ?
>>>
>>> Do you have Mail-SPF perl module installed?
>
>
> --
> Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
> Warning: I wish NOT to receive e-mail advertising to this address.
> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
> "They say when you play that M$ CD backward you can hear satanic messages."
> "That's nothing. If you play it forward it will install Windows."

Re: spf

[Jered Floyd]

Oops; I misread -- Gokan is the one reporting is issue.

Zimbra is bit unique -- they distribute most of the large OS components as they 
have them pinned to various versions.  So this is really going to be an issue 
best addressed by them.

For example, my install (on an Ubuntu 14.04 LTS base) includes these packages 
from Zimbra:

zimbra-perl
zimbra-perl-archive-zip
zimbra-perl-base
zimbra-perl-berkeleydb
zimbra-perl-bit-vector
zimbra-perl-cache-fastmmap
zimbra-perl-canary-stability
zimbra-perl-carp-clan
zimbra-perl-class-inspector
zimbra-perl-compress-raw-bzip2
zimbra-perl-compress-raw-zlib
zimbra-perl-config-inifiles
zimbra-perl-convert-asn1
zimbra-perl-convert-binhex
zimbra-perl-convert-tnef
zimbra-perl-convert-uulib
zimbra-perl-crypt-openssl-random
zimbra-perl-crypt-openssl-rsa
zimbra-perl-crypt-saltedhash
zimbra-perl-data-uuid
zimbra-perl-date-calc
zimbra-perl-date-manip
zimbra-perl-db-file
zimbra-perl-dbd-mysql
zimbra-perl-dbd-sqlite
zimbra-perl-dbi
zimbra-perl-digest-hmac
zimbra-perl-digest-sha1
zimbra-perl-email-date-format
zimbra-perl-encode-detect
zimbra-perl-encode-locale
zimbra-perl-error
zimbra-perl-exporter-tiny
zimbra-perl-file-grep
zimbra-perl-file-libmagic
zimbra-perl-file-listing
zimbra-perl-file-tail
zimbra-perl-filesys-df
zimbra-perl-geography-countries
zimbra-perl-html-parser
zimbra-perl-http-cookies
zimbra-perl-http-daemon
zimbra-perl-http-date
zimbra-perl-http-message
zimbra-perl-http-negotiate
zimbra-perl-innotop
zimbra-perl-io-compress
zimbra-perl-io-html
zimbra-perl-io-sessiondata
zimbra-perl-io-socket-inet6
zimbra-perl-io-socket-ip
zimbra-perl-io-socket-ssl
zimbra-perl-io-stringy
zimbra-perl-ip-country
zimbra-perl-json-pp
zimbra-perl-libwww
zimbra-perl-list-moreutils
zimbra-perl-lwp-mediatypes
zimbra-perl-lwp-protocol-https
zimbra-perl-mail-dkim
zimbra-perl-mail-spamassassin
zimbra-perl-mail-spf
zimbra-perl-mailtools
zimbra-perl-math-bigint
zimbra-perl-mime-lite
zimbra-perl-mime-tools
zimbra-perl-mime-types
zimbra-perl-mozilla-ca
zimbra-perl-net-cidr
zimbra-perl-net-cidr-lite
zimbra-perl-net-dns
zimbra-perl-net-dns-resolver-programmable
zimbra-perl-net-http
zimbra-perl-net-ldap
zimbra-perl-net-ldapapi
zimbra-perl-net-libidn
zimbra-perl-net-server
zimbra-perl-net-ssleay
zimbra-perl-netaddr-ip
zimbra-perl-parent
zimbra-perl-proc-processtable
zimbra-perl-soap-lite
zimbra-perl-socket
zimbra-perl-socket-linux
zimbra-perl-swatchdog
zimbra-perl-task-weaken
zimbra-perl-term-readkey
zimbra-perl-timedate
zimbra-perl-unix-getrusage
zimbra-perl-unix-syslog
zimbra-perl-uri
zimbra-perl-www-robotrules
zimbra-perl-xml-namespacesupport
zimbra-perl-xml-parser
zimbra-perl-xml-parser-lite
zimbra-perl-xml-sax
zimbra-perl-xml-sax-base
zimbra-perl-xml-sax-expat
zimbra-perl-xml-simple
zimbra-perl-zmq-constants
zimbra-perl-zmq-libzmq3
zimbra-spamassassin-rules

Regards,
--Jered

- On Feb 26, 2017, at 10:22 AM, Matus UHLAR - fantomas uh...@fantomas.sk 
wrote:

> On 26.02.17 10:12, Jered Floyd wrote:
>>It may be more effective to pursue this in a Zimbra support forum.
> 
> perl modules more belong to the perl and OS installation, not to zimbra.
> 
> 
> --
> Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
> Warning: I wish NOT to receive e-mail advertising to this address.
> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
> A day without sunshine is like, night.

Re: spf

[Matus UHLAR - fantomas]

On 26.02.17 10:12, Jered Floyd wrote:

It may be more effective to pursue this in a Zimbra support forum.


perl modules more belong to the perl and OS installation, not to zimbra.


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
A day without sunshine is like, night.

Re: spf

[Jered Floyd]

Matus,

It may be more effective to pursue this in a Zimbra support forum.

If you are running the OSS version (as am I!), you may have good luck here: 
https://forums.zimbra.org/

Unfortunately I run SpamAssassin on my mailhub prior to Zimbra so can't help on 
this one.

--Jered

- On Feb 26, 2017, at 7:20 AM, Gokan Atmaca linux.go...@gmail.com wrote:

>> Do you have Mail-SPF perl module installed?
> 
> Hello
> 
> i am using zimbra. The module is being installed.
> 
> /opt/zimbra/data/spamassassin/rules/25_antivirus.cf
> /opt/zimbra/data/spamassassin/rules/25_asn.cf
> /opt/zimbra/data/spamassassin/rules/25_dcc.cf
> /opt/zimbra/data/spamassassin/rules/25_dkim.cf
> /opt/zimbra/data/spamassassin/rules/25_hashcash.cf
> /opt/zimbra/data/spamassassin/rules/25_pyzor.cf
> /opt/zimbra/data/spamassassin/rules/25_razor2.cf
> /opt/zimbra/data/spamassassin/rules/25_replace.cf
> /opt/zimbra/data/spamassassin/rules/25_spf.cf -| SPF module
> /opt/zimbra/data/spamassassin/rules/25_textcat.cf
> /opt/zimbra/data/spamassassin/rules/25_uribl.cf
> /opt/zimbra/data/spamassassin/rules/30_text_de.cf
> /opt/zimbra/data/spamassassin/rules/30_text_fr.cf
> /opt/zimbra/data/spamassassin/rules/30_text_it.cf
> 
> On Sun, Feb 26, 2017 at 2:36 PM, Matus UHLAR - fantomas
>  wrote:
>> On 26.02.17 13:46, Gokan Atmaca wrote:
>>>
>>> I want to check SPF. But it does not work. Can you help me with this?
>>>
>>> config:
>>> header SPF_GECER eval:check_for_spf_pass()
>>> describe SPF_GECER   SPF: sender matches SPF record
>>> score SPF_GECER  -0.5
>>
>>
>> what is wrong with existing and provided SPF_PASS rule (and the others)?
>> did you load loadplugin Mail::SpamAssassin::Plugin::SPF ?
>>
>> Do you have Mail-SPF perl module installed?
>>
>>
>> --
>> Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
>> Warning: I wish NOT to receive e-mail advertising to this address.
>> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
>> Saving Private Ryan...
> > Private Ryan exists. Overwrite? (Y/N)

Re: spf

[Matus UHLAR - fantomas]

Do you have Mail-SPF perl module installed?


On 26.02.17 15:20, Gokan Atmaca wrote:

i am using zimbra. The module is being installed.

/opt/zimbra/data/spamassassin/rules/25_spf.cf -| SPF module


that is not SPF module, nor a perl module. try searching it:

% locate -i spf.pm
/usr/lib/perl5/Net/DNS/RR/SPF.pm
/usr/share/perl5/Mail/SPF.pm   # this is the module.
/usr/share/perl5/Mail/SpamAssassin/Plugin/SPF.pm

you also nees "loadplugin Mail::SpamAssassin::Plugin::SPF" uncommended in
spamassassin's init.pre


On Sun, Feb 26, 2017 at 2:36 PM, Matus UHLAR - fantomas
 wrote:

On 26.02.17 13:46, Gokan Atmaca wrote:


I want to check SPF. But it does not work. Can you help me with this?

config:
header SPF_GECER eval:check_for_spf_pass()
describe SPF_GECER   SPF: sender matches SPF record
score SPF_GECER  -0.5



what is wrong with existing and provided SPF_PASS rule (and the others)?
did you load loadplugin Mail::SpamAssassin::Plugin::SPF ?

Do you have Mail-SPF perl module installed?


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
"They say when you play that M$ CD backward you can hear satanic messages."
"That's nothing. If you play it forward it will install Windows."

Re: spf

[Gokan Atmaca]

> Do you have Mail-SPF perl module installed?

Hello

i am using zimbra. The module is being installed.

/opt/zimbra/data/spamassassin/rules/25_antivirus.cf
/opt/zimbra/data/spamassassin/rules/25_asn.cf
/opt/zimbra/data/spamassassin/rules/25_dcc.cf
/opt/zimbra/data/spamassassin/rules/25_dkim.cf
/opt/zimbra/data/spamassassin/rules/25_hashcash.cf
/opt/zimbra/data/spamassassin/rules/25_pyzor.cf
/opt/zimbra/data/spamassassin/rules/25_razor2.cf
/opt/zimbra/data/spamassassin/rules/25_replace.cf
/opt/zimbra/data/spamassassin/rules/25_spf.cf -| SPF module
/opt/zimbra/data/spamassassin/rules/25_textcat.cf
/opt/zimbra/data/spamassassin/rules/25_uribl.cf
/opt/zimbra/data/spamassassin/rules/30_text_de.cf
/opt/zimbra/data/spamassassin/rules/30_text_fr.cf
/opt/zimbra/data/spamassassin/rules/30_text_it.cf

On Sun, Feb 26, 2017 at 2:36 PM, Matus UHLAR - fantomas
 wrote:
> On 26.02.17 13:46, Gokan Atmaca wrote:
>>
>> I want to check SPF. But it does not work. Can you help me with this?
>>
>> config:
>> header SPF_GECER eval:check_for_spf_pass()
>> describe SPF_GECER   SPF: sender matches SPF record
>> score SPF_GECER  -0.5
>
>
> what is wrong with existing and provided SPF_PASS rule (and the others)?
> did you load loadplugin Mail::SpamAssassin::Plugin::SPF ?
>
> Do you have Mail-SPF perl module installed?
>
>
> --
> Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
> Warning: I wish NOT to receive e-mail advertising to this address.
> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
> Saving Private Ryan...
> Private Ryan exists. Overwrite? (Y/N)

Re: spf

[Matus UHLAR - fantomas]

On 26.02.17 13:46, Gokan Atmaca wrote:

I want to check SPF. But it does not work. Can you help me with this?

config:
header SPF_GECER eval:check_for_spf_pass()
describe SPF_GECER   SPF: sender matches SPF record
score SPF_GECER  -0.5


what is wrong with existing and provided SPF_PASS rule (and the others)?
did you load loadplugin Mail::SpamAssassin::Plugin::SPF ?

Do you have Mail-SPF perl module installed?


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Saving Private Ryan...
Private Ryan exists. Overwrite? (Y/N)

Re: SPF PermError or TempError cannot hit

[Merijn van den Kroonenberg]

> I realized that the rules T_SPF_PERMERROR and T_SPF_TEMPERROR were never
> hitting on my emails even though my Postfix log had multiple instances
> of such errors, e.g. this timeout

Hmm, thats weird, they hit just fine over here...

>
> 2017-01-16 14:03:35-0500 [postfix] 10111.5ms ip=173.37.142.90
> h=alln-iport-3.cisco.comfrom=p...@cisco.com
> to=u...@domain.com  -> PREPEND Received-SPF: TempError (u...@domain.com:
> temporary error in processing during lookup
> of cisco.com) client-ip=173.37.142.90; envelope-from="p...@cisco.com";
> helo=alln-iport-3.cisco.com;
> receiver=u...@domain.com; identity=mailfrom
>
> I did a bit of digging into the code and found that line 394 of the SPF
> plugin
> 
> checks for valid Received-SPF headers
>
>  if ($hdr =~
> /^received-spf:\s*(pass|neutral|(?:soft)?fail|none)\b(?:.*\bidentity=(\S+?);?\b)?/i)
> {
>
> Since /TempError/ and /PermError/ are not handled, the SPF is never
> checked. Editing this regex to include them fixes the problem.
>
> Another aspect I find surprising is that T_SPF_HELO_PERMERROR and
> T_SPF_HELO_TEMPERROR do hit regularly. My hypothesis is there is an
> actual DNS lookup by the SPF module is used instead of the headers, but
> I did not dig into the code enough to find out.
>
> Is this an issue that other people have experienced? I am using
> spamassassin 3.4.1 and sa-update version svn1652181
>

Are you sure you run 3.4.1? It looks like there were some changes related
to temperror and permerror between 3.4.0 and 3.4.1.


> --
> Olivier Coutu
>
>

Re: SPF should always hit? SOLVED

[Reindl Harald]

Am 11.07.2016 um 21:02 schrieb David B Funk:

On Mon, 11 Jul 2016, Reindl Harald wrote:

SA has also a weakness or design mistake here

"envelope_sender_header X-Local-Envelope-From" while that header comes
from postfix with customized configuration because we use it in own
rules has no fallback
__

By default, various MTAs will use different headers, such as the
following:

   X-Envelope-From
   Envelope-Sender
   X-Sender
   Return-Path
__

well, in case of "envelope_sender_header" present in the configuration
and that header is missing for whatever reason there is *no fallback*
while for most cases it would be better to use
"envelope_sender_header" as prefered one instead the only one

that it is not the case can you see when "add_header all Status
_YESNO_, score=_SCORE_, tag-level=_REQD_, block-level=8.0,
envelope=_SENDERDOMAIN_, from=_AUTHORDOMAIN_, _TOKENSUMMARY_" ends
with SENDERDOMAIN_ in your headers


The SA Conf man page seems to indicate that it -should- fall back to its
heuristic if the envelope_sender_header is missing:

   To avoid this heuristic failure, the "envelope_sender_header" setting
may be helpful.  Name
   the header that your MTA or MDA adds to messages containing the
address used at the MAIL
   FROM step of the SMTP transaction.

   If the header in question contains "<" or ">" characters at the start
and end of the email
   address in the right-hand side, as in the SMTP transaction, these
will be stripped.

   If the header is not found in a message, or if it's value does not
contain an "@" sign,
   SpamAssassin will issue a warning in the logs and fall back to its
default heuristics.

It doesn't look like that fall-back is working. If you completely omit
the envelope_sender_header config setting, the heuristic works.
Maybe you should file a bug-report.


looks so


One additional question, if you're setting the envelope_sender_header
configwhy aren't you actually supplying it?


because i have *no idea* from where it comes that postfix sometimes ignores

 check_sender_access proxy:pcre:/etc/postfix/x_envelope_from.cf
 check_recipient_access proxy:pcre:/etc/postfix/x_envelope_to.cf


If you cannot depend upon your system to actually supply the header you
list
in your envelope_sender_header config, then don't set that parameter


well, the idea is to add a own heaer in the MTA instead rely on 
heuristic which hopefully don't use a randm but wrong header (if that 
would be impossible the other problem also won't exist)




signature.asc
Description: OpenPGP digital signature

Re: SPF should always hit? SOLVED

[David B Funk]

On Mon, 11 Jul 2016, Reindl Harald wrote:



Am 11.07.2016 um 19:30 schrieb RW:

[snip..]

It sounds like SA is not able to parse the envelope sender out of the
headers.

See the description for envelope_sender_header in
man Mail::SpamAssassin::Conf


SA has also a weakness or design mistake here

"envelope_sender_header X-Local-Envelope-From" while that header comes from 
postfix with customized configuration because we use it in own rules has no 
fallback

__

By default, various MTAs will use different headers, such as the following:

   X-Envelope-From
   Envelope-Sender
   X-Sender
   Return-Path
__

well, in case of "envelope_sender_header" present in the configuration and 
that header is missing for whatever reason there is *no fallback* while for 
most cases it would be better to use "envelope_sender_header" as prefered one 
instead the only one


that it is not the case can you see when "add_header all Status _YESNO_, 
score=_SCORE_, tag-level=_REQD_, block-level=8.0, envelope=_SENDERDOMAIN_, 
from=_AUTHORDOMAIN_, _TOKENSUMMARY_" ends with SENDERDOMAIN_ in your headers


The SA Conf man page seems to indicate that it -should- fall back to its 
heuristic if the envelope_sender_header is missing:


   To avoid this heuristic failure, the "envelope_sender_header" setting may be 
helpful.  Name
   the header that your MTA or MDA adds to messages containing the address used 
at the MAIL
   FROM step of the SMTP transaction.

   If the header in question contains "<" or ">" characters at the start and 
end of the email
   address in the right-hand side, as in the SMTP transaction, these will be 
stripped.

   If the header is not found in a message, or if it's value does not contain an 
"@" sign,
   SpamAssassin will issue a warning in the logs and fall back to its default 
heuristics.

It doesn't look like that fall-back is working. If you completely omit the 
envelope_sender_header config setting, the heuristic works.

Maybe you should file a bug-report.

One additional question, if you're setting the envelope_sender_header config
why aren't you actually supplying it?

If you cannot depend upon your system to actually supply the header you list
in your envelope_sender_header config, then don't set that parameter.

--
Dave Funk  University of Iowa
College of Engineering
319/335-5751   FAX: 319/384-0549   1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include 
Better is not better, 'standard' is better. B{

smime.p7s
Description: S/MIME Cryptographic Signature

Re: SPF should always hit? SOLVED

[Reindl Harald]

Am 11.07.2016 um 19:30 schrieb RW:

On Mon, 11 Jul 2016 12:49:04 -0400
Robert Fitzpatrick wrote:


I finally was able to get SPF checks to be more reliable by making
sure Postfix SPF policies were in place. Here is a good read 

https://github.com/mail-in-a-box/mailinabox/issues/698
Excerpt: It's worth noting that lack of postfix's spf checker renders
spamassassin's flagging impaired because without it spamassassin in
my case is only adding helo_pass and that's all regarding spfs.


It sounds like SA is not able to parse the envelope sender out of the
headers.

See the description for envelope_sender_header in
man Mail::SpamAssassin::Conf


SA has also a weakness or design mistake here

"envelope_sender_header X-Local-Envelope-From" while that header comes 
from postfix with customized configuration because we use it in own 
rules has no fallback

__

By default, various MTAs will use different headers, such as the following:

X-Envelope-From
Envelope-Sender
X-Sender
Return-Path
__

well, in case of "envelope_sender_header" present in the configuration 
and that header is missing for whatever reason there is *no fallback* 
while for most cases it would be better to use "envelope_sender_header" 
as prefered one instead the only one


that it is not the case can you see when "add_header all Status _YESNO_, 
score=_SCORE_, tag-level=_REQD_, block-level=8.0, 
envelope=_SENDERDOMAIN_, from=_AUTHORDOMAIN_, _TOKENSUMMARY_" ends with 
SENDERDOMAIN_ in your headers




signature.asc
Description: OpenPGP digital signature

Re: SPF should always hit? SOLVED

[RW]

On Mon, 11 Jul 2016 12:49:04 -0400
Robert Fitzpatrick wrote:

> I finally was able to get SPF checks to be more reliable by making
> sure Postfix SPF policies were in place. Here is a good read 
> 
> https://github.com/mail-in-a-box/mailinabox/issues/698
> Excerpt: It's worth noting that lack of postfix's spf checker renders 
> spamassassin's flagging impaired because without it spamassassin in
> my case is only adding helo_pass and that's all regarding spfs.


It sounds like SA is not able to parse the envelope sender out of the
headers.

See the description for envelope_sender_header in
man Mail::SpamAssassin::Conf

Re: SPF should always hit? SOLVED

[Robert Fitzpatrick]

Robert Fitzpatrick wrote:

Joe Quinn wrote:

On 6/9/2016 11:23 AM, Robert Fitzpatrick wrote:

Excuse me if this is too lame a question, but I have the SPF plugin
enabled and it hits a lot. Should SPF_ something hit on every message
if the domain has an SPF record in DNS?

Furthermore, a message found as Google phishing did not get a hit on a
email address where the domain has SPF setup. Not sure if it would
fail anyway if the envelope from is the culprit?


In a perfect world, every message you scan will hit one of the following:
SPF_HELO_NONE
SPF_HELO_NEUTRAL
SPF_HELO_PASS
SPF_HELO_FAIL
SPF_HELO_SOFTFAIL
T_SPF_HELO_PERMERROR
T_SPF_HELO_TEMPERROR

And additionally one of the following:
SPF_NONE
SPF_NEUTRAL
SPF_PASS
SPF_FAIL
SPF_SOFTFAIL
T_SPF_PERMERROR
T_SPF_TEMPERROR



I finally was able to get SPF checks to be more reliable by making sure 
Postfix SPF policies were in place. Here is a good read 


https://github.com/mail-in-a-box/mailinabox/issues/698
Excerpt: It's worth noting that lack of postfix's spf checker renders 
spamassassin's flagging impaired because without it spamassassin in my 
case is only adding helo_pass and that's all regarding spfs.


Once we got Postfix SPF checks setup using the Python version and 
disabling rejects in the config, we now have headers we can be sure are 
handled by our custom rules in addition to any SA checks.


--
Robert

Re: SPF should always hit?

[Robert Fitzpatrick]

Joe Quinn wrote:

On 6/9/2016 11:23 AM, Robert Fitzpatrick wrote:

Excuse me if this is too lame a question, but I have the SPF plugin
enabled and it hits a lot. Should SPF_ something hit on every message
if the domain has an SPF record in DNS?

Furthermore, a message found as Google phishing did not get a hit on a
email address where the domain has SPF setup. Not sure if it would
fail anyway if the envelope from is the culprit?


In a perfect world, every message you scan will hit one of the following:
SPF_HELO_NONE
SPF_HELO_NEUTRAL
SPF_HELO_PASS
SPF_HELO_FAIL
SPF_HELO_SOFTFAIL
T_SPF_HELO_PERMERROR
T_SPF_HELO_TEMPERROR

And additionally one of the following:
SPF_NONE
SPF_NEUTRAL
SPF_PASS
SPF_FAIL
SPF_SOFTFAIL
T_SPF_PERMERROR
T_SPF_TEMPERROR

In practice, there's almost certainly a few edge cases where messages
can avoid getting one in either category. For purposes of writing your
own metas against these, the rules that matter most for measuring
spamminess are the none, pass, and fail/softfail results. The rest are
for total coverage of the results that an SPF query can yield, for
debugging and documentation purposes.

Also, none of these will hit at all if you disable network tests.


Yes, network tests are on. I have lots of messages hitting, it is harder 
to find one that doesn't have hits as you suggested. However, I can find 
several out of our database of 280K messages cached which do not hit any 
of these rules. So, what would be a reason they didn't hit?


The only custom rule I have with SPF_* is with SPF_FAIL combined without 
DKIM to give higher score:


meta WT_FORGED_SENDER (SPF_FAIL && !DKIM_VALID)
describe WT_FORGED_SENDER To score high when SPF fails without valid DKIM
scoreWT_FORGED_SENDER 8.0

Here is the score for this particular example:

2.095   FREEMAIL_FORGED_REPLYTO Freemail in Reply-To, but not From
1.000   XPRIO_SHORT_SUBJ(No description provided)
0.250   FREEMAIL_REPLYTO_END_DIGIT  Reply-To freemail username ends in digit
0.001   HTML_MESSAGEHTML included in message
0.001   HEADER_FROM_DIFFERENT_DOMAINS   (No description provided)
0.000   RCVD_IN_DNSWL_NONE  Sender listed at http://www.dnswl.org/, low 
trust
-1.900  BAYES_00Bayesian spam probability is 0 to 1%
-5.000  RCVD_IN_JMF_W   (No description provided)

--
Robert

Re: SPF should always hit?

[Joe Quinn]

On 6/9/2016 11:23 AM, Robert Fitzpatrick wrote:
Excuse me if this is too lame a question, but I have the SPF plugin 
enabled and it hits a lot. Should SPF_ something hit on every message 
if the domain has an SPF record in DNS?


Furthermore, a message found as Google phishing did not get a hit on a 
email address where the domain has SPF setup. Not sure if it would 
fail anyway if the envelope from is the culprit?



In a perfect world, every message you scan will hit one of the following:
SPF_HELO_NONE
SPF_HELO_NEUTRAL
SPF_HELO_PASS
SPF_HELO_FAIL
SPF_HELO_SOFTFAIL
T_SPF_HELO_PERMERROR
T_SPF_HELO_TEMPERROR

And additionally one of the following:
SPF_NONE
SPF_NEUTRAL
SPF_PASS
SPF_FAIL
SPF_SOFTFAIL
T_SPF_PERMERROR
T_SPF_TEMPERROR

In practice, there's almost certainly a few edge cases where messages 
can avoid getting one in either category. For purposes of writing your 
own metas against these, the rules that matter most for measuring 
spamminess are the none, pass, and fail/softfail results. The rest are 
for total coverage of the results that an SPF query can yield, for 
debugging and documentation purposes.


Also, none of these will hit at all if you disable network tests.

Re: SPF should always hit?

[Reindl Harald]

Am 09.06.2016 um 17:23 schrieb Robert Fitzpatrick:

Excuse me if this is too lame a question, but I have the SPF plugin
enabled and it hits a lot. Should SPF_ something hit on every message if
the domain has an SPF record in DNS?


and if it's SPF_NONE


Furthermore, a message found as Google phishing did not get a hit on a
email address where the domain has SPF setup. Not sure if it would fail
anyway if the envelope from is the culprit?


SPF is always about envelopes (no trolling about sender-id needed this 
time) and with the infos you provided what do you expect for answers?


just read https://en.wikipedia.org/wiki/Sender_Policy_Framework

SPF is more than just "fail"

T_SPF_PERMERROR
T_SPF_TEMPERROR
T_SPF_HELO_PERMERROR
T_SPF_HELO_TEMPERROR
SPF_FAIL
SPF_HELO_FAIL
SPF_HELO_NEUTRAL
SPF_HELO_NONE
SPF_HELO_PASS
SPF_HELO_SOFTFAIL
SPF_NEUTRAL
SPF_NONE
SPF_PASS
SPF_SOFTFAIL




signature.asc
Description: OpenPGP digital signature

Re: SPF rules and my domain

[Matus UHLAR - fantomas]

On 10.12.15 22:54, Alex wrote:

I don't understand why a message from tripadvisor.com would have
SPF_FAIL, and as part of trying to understand how SPF works, I'd like
to figure out what's happening.

Would someone be able to take a look at this message and figure out
why mail from tripadvisor.com fails SPF?

http://pastebin.com/36hzGcTs


On 11.12.15 08:56, Matus UHLAR - fantomas wrote:

the envelope sender seems to be
bounce-15_html-74319930-51788793-10834732...@bounce.e.tripadvisor.com

bounce.e.tripadvisor.com seems to have no SPF record, so I also don't
understand why SPF tests should hit at all, maybe SPF HELO tests...


disregard, please. I made an mistype when checking the SPF records.


The main reason why the mail hits SPF_FAIL is that you don't trust even servers
you receive mail from - first three hops:

h02p01.smtp.routit.net (h02p01.smtp.routit.net [89.146.30.9])
pop3.routit.net ([213.144.235.7])
h03p02.smtp.routit.net (h03p02.smtp.routit.net [89.146.30.18])

that are in X-Spam-RelaysUntrusted header.

the next server in path is:
mta3.e.tripadvisor.com ([66.231.81.9])

that passes the SPF test.

That simply indicates error in yout trust path
http://wiki.apache.org/spamassassin/TrustPath




--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
There's a long-standing bug relating to the x86 architecture that
allows you to install Windows.   -- Matthew D. Fuller

Re: SPF rules and my domain

[Reindl Harald]

Am 11.12.2015 um 17:11 schrieb Alex:

On Fri, Dec 11, 2015 at 10:33 AM, Matus UHLAR - fantomas
 wrote:

On 10.12.15 22:54, Alex wrote:


I don't understand why a message from tripadvisor.com would have
SPF_FAIL, and as part of trying to understand how SPF works, I'd like
to figure out what's happening.

Would someone be able to take a look at this message and figure out
why mail from tripadvisor.com fails SPF?

http://pastebin.com/36hzGcTs


On 11.12.15 08:56, Matus UHLAR - fantomas wrote:


the envelope sender seems to be
bounce-15_html-74319930-51788793-10834732...@bounce.e.tripadvisor.com

bounce.e.tripadvisor.com seems to have no SPF record, so I also don't
understand why SPF tests should hit at all, maybe SPF HELO tests...


disregard, please. I made an mistype when checking the SPF records.

The main reason why the mail hits SPF_FAIL is that you don't trust even
servers
you receive mail from - first three hops:

h02p01.smtp.routit.net (h02p01.smtp.routit.net [89.146.30.9])
pop3.routit.net ([213.144.235.7])
h03p02.smtp.routit.net (h03p02.smtp.routit.net [89.146.30.18])


Is it possible this message was forwarded, breaking the trust path?


when you try to understand a little bit what SPF does the answer is 
clearly yes, any Received header not litest in your trusted_networks or 
internal_networks after the origin server delivered the mail will not 
break SPF but also *every* DNSBL/DNSWL and so fire all sort of 
false-postives as well as false negatives



Reindl wrote:

who is that?
Received: from h03p02.smtp.routit.net (h03p02.smtp.routit.net [89.146.30.18]) 
by pop3.routit.net (Postfix)
with ESMTP id D0A1B42463
for ; Fri, 11 Dec 2015 02:18:21 +0100 (CET)

who is that?
Received: from pop3.routit.net ([213.144.235.7]) by h02p01.smtp.routit.net with 
ESMTP; 11
Dec 2015 02:18:26 +0100

who is that?
Received: from h02p01.smtp.routit.net (h02p01.smtp.routit.net [89.146.30.9]) by 
bwimail02.example.com
(Postfix) with ESMTP id 0F8CA345F25 for ; Thu, 10 
Dec
2015 20:18:38 -0500 (EST)


We're not responsible for the example.nl domain


man that message has *five* Received headers after the tripadvsior 
server - and it smells like a combination of forwarding and fetchmail


Received: from pop3.routit.net ([213.144.235.7]) by 
h02p01.smtp.routit.net with ESMTP; 11 Dec 2015 02:18:26 +0100


Received: from h03p02.smtp.routit.net (h03p02.smtp.routit.net 
[89.146.30.18]) by pop3.routit.net (Postfix) with ESMTP id D0A1B42463 
for ; Fri, 11 Dec 2015 02:18:21 +0100 (CET)



so I don't understand
where that came from. Perhaps that account forwarded it to the
wytze.vandenb...@example.com (our domain) without any other
indications of that having occurred, breaking the trust path?


surely


that are in X-Spam-RelaysUntrusted header.

the next server in path is:
mta3.e.tripadvisor.com ([66.231.81.9])

that passes the SPF test.


What did you need to do to test this?


what do you need to test when "mta3.e.tripadvisor.com" is clearly 
"tripadvisor.com" and that received header is burried in the middle of 
other received headers?




signature.asc
Description: OpenPGP digital signature

Re: SPF rules and my domain

[Alex]

Hi,

On Fri, Dec 11, 2015 at 10:33 AM, Matus UHLAR - fantomas
 wrote:
>> On 10.12.15 22:54, Alex wrote:
>>>
>>> I don't understand why a message from tripadvisor.com would have
>>> SPF_FAIL, and as part of trying to understand how SPF works, I'd like
>>> to figure out what's happening.
>>>
>>> Would someone be able to take a look at this message and figure out
>>> why mail from tripadvisor.com fails SPF?
>>>
>>> http://pastebin.com/36hzGcTs
>
>
> On 11.12.15 08:56, Matus UHLAR - fantomas wrote:
>>
>> the envelope sender seems to be
>> bounce-15_html-74319930-51788793-10834732...@bounce.e.tripadvisor.com
>>
>> bounce.e.tripadvisor.com seems to have no SPF record, so I also don't
>> understand why SPF tests should hit at all, maybe SPF HELO tests...
>
> disregard, please. I made an mistype when checking the SPF records.
>
> The main reason why the mail hits SPF_FAIL is that you don't trust even
> servers
> you receive mail from - first three hops:
>
> h02p01.smtp.routit.net (h02p01.smtp.routit.net [89.146.30.9])
> pop3.routit.net ([213.144.235.7])
> h03p02.smtp.routit.net (h03p02.smtp.routit.net [89.146.30.18])

Is it possible this message was forwarded, breaking the trust path?

Reindal wrote:
> who is that?
> Received: from h03p02.smtp.routit.net (h03p02.smtp.routit.net [89.146.30.18]) 
> by pop3.routit.net (Postfix)
> with ESMTP id D0A1B42463
>for ; Fri, 11 Dec 2015 02:18:21 +0100 (CET)
>
> who is that?
> Received: from pop3.routit.net ([213.144.235.7]) by h02p01.smtp.routit.net 
> with ESMTP; 11
> Dec 2015 02:18:26 +0100
>
> who is that?
> Received: from h02p01.smtp.routit.net (h02p01.smtp.routit.net [89.146.30.9]) 
> by bwimail02.example.com
> (Postfix) with ESMTP id 0F8CA345F25 for ; Thu, 
> 10 Dec
> 2015 20:18:38 -0500 (EST)

We're not responsible for the example.nl domain, so I don't understand
where that came from. Perhaps that account forwarded it to the
wytze.vandenb...@example.com (our domain) without any other
indications of that having occurred, breaking the trust path?

> that are in X-Spam-RelaysUntrusted header.
>
> the next server in path is:
> mta3.e.tripadvisor.com ([66.231.81.9])
>
> that passes the SPF test.

What did you need to do to test this?

Thanks again,
Alex

Re: SPF rules and my domain

[Reindl Harald]

Am 11.12.2015 um 08:56 schrieb Matus UHLAR - fantomas:

I don't understand why a message from tripadvisor.com would have
SPF_FAIL, and as part of trying to understand how SPF works, I'd like
to figure out what's happening.

Would someone be able to take a look at this message and figure out
why mail from tripadvisor.com fails SPF?

http://pastebin.com/36hzGcTs


the envelope sender seems to be
bounce-15_html-74319930-51788793-10834732...@bounce.e.tripadvisor.com

bounce.e.tripadvisor.com seems to have no SPF record, so I also don't
understand why SPF tests should hit at all, maybe SPF HELO tests...


how dou you come to that conclusion and what means "seems" in that 
context - there is a TXT record or there is none - it's has one


;; ANSWER SECTION:
bounce.e.tripadvisor.com. 3600  IN  TXT "v=spf1 
include:cust-spf.exacttarget.com -all"


there is a ton of recievd headres and pretty sure spamassassin is *not* 
running on the MTA and internal_netwroks / trusted_networks is not 
configured properly at all and in that case DNSBL/DNSWL are also not 
wokring properly


that is the last external hop
Received: from mta3.e.tripadvisor.com ([66.231.81.9])
by h03p02.smtp.routit.net with ESMTP; 11 Dec 2015 02:18:24 +0100
_

who is that?
Received: from h03p02.smtp.routit.net (h03p02.smtp.routit.net 
[89.146.30.18]) by pop3.routit.net (Postfix) with ESMTP id D0A1B42463

for ; Fri, 11 Dec 2015 02:18:21 +0100 (CET)

who is that?
Received: from pop3.routit.net ([213.144.235.7]) by 
h02p01.smtp.routit.net with ESMTP; 11 Dec 2015 02:18:26 +0100


who is that?
Received: from h02p01.smtp.routit.net (h02p01.smtp.routit.net 
[89.146.30.9]) by bwimail02.example.com (Postfix) with ESMTP id 
0F8CA345F25 for ; Thu, 10 Dec 2015 
20:18:38 -0500 (EST)




signature.asc
Description: OpenPGP digital signature

Re: SPF rules and my domain

[Reindl Harald]

Am 10.12.2015 um 03:42 schrieb Alex:

If I wanted to use SPF in spamassassin to block spoofing attempts
against my domain, how would I do that?

Can I create a meta that combines SPF_FAIL with the From header for my
domain to do this?


SPF *is not* about the From-Header



signature.asc
Description: OpenPGP digital signature

Re: SPF rules and my domain

[Matus UHLAR - fantomas]

Yes, understood. This was always about my own MTA receiving a message
appearing to be "FROM" my own domain, and my own SPF record would be
used to check the IP of the remote system to determine if it was
permitted. I may have made that especially clear at one point.

Does this make sense now? I'm trying to use my SPF record to verify
mail FROM our domain being received by our MX is not spoofed.


Right, that was understood.

My response was based on how you worded your question, which has been
removed from the thread now:


> > Please help me understand why SPF_FAIL would not be triggered when > >
> > an incoming email using my domain is received by a server that is > > not in
> > my SPF record.


The SPF fail SHOULD be triggered in that case. But in your first mail you
have mentioned T_SPF_PERMERROR hits instead and later you have denied it...

it's hard to explain why it does not work if you don't provide proper info.


I was addressing the apparent assumption within that question that the
recipient MTA matters to SPF validation.


On 09.12.15 21:42, Alex wrote:

I'm not sure if there's a question there, or I'm still confused. It
matters because the recipient MTA is my own.

Spamassassin is just going to record a generic SPF_FAIL, regardless of
whether it's my SPF record or an email from some other domain.

If I wanted to use SPF in spamassassin to block spoofing attempts
against my domain, how would I do that?


If anyone tried to send mail _to_ your MTA spoofing _your_ domain from
_remote_ (not your) IP that is apparently _not_ part of your domain's SPF
recors, your MTA will refuse the mail as SPF_FAIL.

That's exactly what you want, isn't it?

The problem you may encounter is just the opposite: legitimate clients using
your MTA should not be refused.  However they should use SMTP Authentication
and that should be prevented from SPF checks.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Quantum mechanics: The dreams stuff is made of. 

Re: SPF rules and my domain

[Benny Pedersen]

On December 10, 2015 3:49:56 PM Alex  wrote:


whitelist_from_spf: *@example.tld (your domain)
header Return-Path =~ example.tld


That's great. I'll investigate.



or blacklist_from *@* with whitelist_auth *@* to hate all equal :)

Re: SPF rules and my domain

[Kris Deugau]

Benny Pedersen wrote:
> Alex skrev den 2015-12-10 03:42:
> 
>> If I wanted to use SPF in spamassassin to block spoofing attempts
>> against my domain, how would I do that?
>> Can I create a meta that combines SPF_FAIL with the From header for my
>> domain to do this?
> 
> setup pypolicyd-spf is not that hard is it ?
> 
> when done, you just configure the spamassassin plugin to reuse the
> recieved-spf header
> 
> data in spf must be with all mynetworks in postfix except all non
> routeble ips such as rfc1918 in the spf for mydestination and virtual
> domains

No, that's not correct.

Postfix $mynetworks, and the equivalent setting in other MTA software,
lists IP ranges that can use your server as an outgoing relay, for any
sender/recipient pair, without further authentication (let's leave aside
any further policy limits you might want that go in other settings;
this is the basic minimum).

An ISP like the one I work for lists "many" IP ranges in $mynetworks
(we're up to ~15 ranges totalling something like an aggregate /15;
essentially all IP address space we've been assigned from ARIN), because
we want to allow our customers to send out their email through our server.

Most of these IP ranges should NOT be emitting any SMTP traffic to the
Internet at large, at all, period;  they should be using the ISP relay
host or some other authenticated third party mail relay server.  So most
of these IPs are irrelevant for SPF except as failure cases.

SPF lists IPs or IP ranges that may use a particular domain as their
envelope sender.

Our SPF record lists a much MUCH smaller list of IPs and ranges;
essentially, the IP ranges our core mail servers live in.  Those are,
formally speaking, the only IP addresses in the world authorized to use
vianet.ca as their SMTP envelope sender.  Third parties should never see
traffic with a vianet.ca envelope sender directly from any other IP.

Note that our customers are authorized by using our outbound relayhost;
 the third party should not "see" the original sender's connection IP
when doing the SPF check.

There is no requirement that there be any overlap between the two,
although in most cases the SPF list is likely a small subset of $mynetworks.

-kgd