Re: Ensuring SPF/DKIM for @gmail.com
On 2023-07-26 at 23:01:11 UTC-0400 (Thu, 27 Jul 2023 13:01:11 +1000) Noel Butler is rumored to have said: On 27/07/2023 10:20, Matija Nalis wrote: [...] Also, 1990s? Weren't first SPF-alike ideas drafted first time in early-mid 2000s, and SPF itself not published as *proposed* IETF standard until 2014? That was less than a decade ago, barely yesterday :) No, SPF pre dates that, 1998 or there abouts if my ageing memory serves me It's failing... :) SPF originated with an idea of Gordon Fecyk, first written up AFTER he left MAPS in 2001. First ID calling it SPF would have been 2003 or so. Paul Vixie and I had both posted about how a 'reverse MX' or 'Mail Sender' record in DNS might work prior to that (and Paul credited someone else with the original idea) but those were not fully-developed mechanisms that anyone could actually deploy. -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) Not Currently Available For Hire
Re: Ensuring SPF/DKIM for @gmail.com
On 27/07/2023 10:20, Matija Nalis wrote: mailing lists have been smart enough for over 20 years to rewrite sender and not appear as a basic forwarder - which are you are correct, however there are forwarding abilities to rewrite sender which avoids this, its been 15 years or more since I've used procmail which by default did not. I personally know several people who still use procmail today, sooo... Your assumption seems to be that EVERYBODY upgrades on regular (yearly-or-so?) cycles, and updates their configs to latest recommended practices at the same time. This is ideal but reality is far different, that said, most would not be using anything from 1990's, if they are, they are have far bigger issues than SPF. That at least I can attest is not always the case (I still see systems with custom sendmail.cf which nobody dares to touch, and with a good reason!) As above. But I won't agree that "it does not exist", nor would I agree that it doesn't matter (if it didn't matter to them, people wouldn't be asking me to troubleshoot it, and yet they do) It "does, not matter", you can't help those who wont help themselves, I'm sure we all remember this back in days when banks and governments wouldt run compliant DNS, they all expected us to whitelist them, when they realised that was not going to happen en masse, they got their act together and fixed their stuff, now, at least in this country, they woke up and realised the benefits so much so, the govt here is a strong proponent of DMARC and mandates all federal govt depts to use it (though I've discovered some that dont) Good for you. But that is anecdotal - you are certainly not participating in every mailing list in existence, I'm on 117 mailing lists - not that I have time these days to read much of it, family life is more important, in past couple weeks I just found a few hours to peruse some :) So, still in 2023, I have to deal with SPF (and DKIM) failing due to such forwarders/ML (as well as misconfigurations, of course) DKIM is a total failure with mailing lists, but DKIM - unlike SPF in a typical setup, is not an out-right reject at MTA level. Also, 1990s? Weren't first SPF-alike ideas drafted first time in early-mid 2000s, and SPF itself not published as *proposed* IETF standard until 2014? That was less than a decade ago, barely yesterday :) No, SPF pre dates that, 1998 or there abouts if my ageing memory serves me correct, 2014 might have been the SPF RR type, which certain cretins from the debian world fought long hard against as their dist versions of bind didnt understand it it was that old (heaven forbid debian users ran modern software - I hope thats changed since but somehow I suspect not...) -- Regards, Noel Butler This Email, including attachments, may contain legally privileged information, therefore at all times remains confidential and subject to copyright protected under international law. You may not disseminate this message without the authors express written authority to do so. If you are not the intended recipient, please notify the sender then delete all copies of this message including attachments immediately. Confidentiality, copyright, and legal privilege are not waived or lost by reason of the mistaken delivery of this message.
Re: Ensuring SPF/DKIM for @gmail.com
On 7/26/23 7:20 PM, Matija Nalis wrote: I'd appreciate more civil expressions of disagreement +1 I personally know several people who still use procmail today, sooo... +1 That at least I can attest is not always the case (I still see systems with custom sendmail.cf which nobody dares to touch, and with a good reason!) Is said sendmail.cf based on a sendmail.mc file or is it older / bespoke? Yeah, I agree that it sure would be nice if world worked that way and everybody upgraded regularly and configured them according to latest BCPs, but around here at least, it sometimes (I'm avoiding to say "often") doesn't. I agree with following the latest BCPs *if* *reasonably* *possible* *to* *do* *so*. But I don't agree with upgrading just because something is old, especially if it's still working. After all, Ethernet is 50 (?) years old and TCP/IP is no spring chicken. Yet we are still using them. There are quite a few systems that someone knowledgable setup some time back, and after they've gone to greener pastures, nobody have touched them, yet they continue to use them. Sometimes you find the diamond in the rough that knows how to care and feed said systems and does tweak them to abide by / support BCPs. Sure, I'll be first to agree that it is bad and should be fixed. What is the actual problem that /needs/ /to/ /be/ /fixed/? Procmail can forward emails without re-writing the envelope if the MTA does the envelope re-writing for it. Just because something is old and doesn't provide the latest and greatest feature doesn't mean that something else can't assist and provide said feature. The complete solution needs to provide the features. Not all features need to be provided by specific components. Telnet over a VPN is perfectly fine especially if the destination doesn't support something better. But I won't agree that "it does not exist", nor would I agree that it doesn't matter (if it didn't matter to them, people wouldn't be asking me to troubleshoot it, and yet they do) #truth +1 Good for you. But that is anecdotal - you are certainly not participating in every mailing list in existence, nor do you contact all people on the planet which use every kind of mail forwarder. Neither do I, but I service lots of systems of other people that do, and with many people, the chances rise. So, still in 2023, I have to deal with SPF (and DKIM) failing due to such forwarders/ML (as well as misconfigurations, of course) +1 Also, 1990s? Weren't first SPF-alike ideas drafted first time in early-mid 2000s, and SPF itself not published as *proposed* IETF standard until 2014? I was wondering, but I couldn't be bothered to look up the dates. I think I started using SPF in the mid-2000s. Maybe late 2000s. That was less than a decade ago, barely yesterday :) LOL Grant. . . .
Re: Ensuring SPF/DKIM for @gmail.com
On Thu, Jul 27, 2023 at 07:11:59AM +1000, Noel Butler wrote: > On 27/07/2023 05:09, Matija Nalis wrote: > > > Any SPF, no matter how correctly configured, will lead to false > > positives in some cases (e.g. encoutering mailing list > > B.S. I'd appreciate more civil expressions of disagreement, though, if this means what I think it means. > mailing lists have been smart enough for over 20 years to rewrite sender and > not appear as a basic forwarder - which are you are correct, however there > are forwarding abilities to rewrite sender which avoids this, its been 15 > years or more since I've used procmail which by default did not. I personally know several people who still use procmail today, sooo... Your assumption seems to be that EVERYBODY upgrades on regular (yearly-or-so?) cycles, and updates their configs to latest recommended practices at the same time. That at least I can attest is not always the case (I still see systems with custom sendmail.cf which nobody dares to touch, and with a good reason!) Yeah, I agree that it sure would be nice if world worked that way and everybody upgraded regularly and configured them according to latest BCPs, but around here at least, it sometimes (I'm avoiding to say "often") doesn't. There are quite a few systems that someone knowledgable setup some time back, and after they've gone to greener pastures, nobody have touched them, yet they continue to use them. Sure, I'll be first to agree that it is bad and should be fixed. But I won't agree that "it does not exist", nor would I agree that it doesn't matter (if it didn't matter to them, people wouldn't be asking me to troubleshoot it, and yet they do) > If you are going to dry-reach to support an argument, please use modern I'm not aware of that "dry-reach" idiom, would you care to explain? > facts and not 1990's. I was a *very* early adopter of SPF back in late 90's > and have had zero issues in 20 years in using SPF (as expected as an early > adopter, teething issues as with all software needed fine tuning in very > early days) Good for you. But that is anecdotal - you are certainly not participating in every mailing list in existence, nor do you contact all people on the planet which use every kind of mail forwarder. Neither do I, but I service lots of systems of other people that do, and with many people, the chances rise. So, still in 2023, I have to deal with SPF (and DKIM) failing due to such forwarders/ML (as well as misconfigurations, of course) Also, 1990s? Weren't first SPF-alike ideas drafted first time in early-mid 2000s, and SPF itself not published as *proposed* IETF standard until 2014? That was less than a decade ago, barely yesterday :) -- Opinions above are GNU-copylefted.
Re: Ensuring SPF/DKIM for @gmail.com
On 7/26/23 2:09 PM, Matija Nalis wrote: Only way to make SPF never incorrectly fail/softwail is to use "+all", but that kind of kills its point :-) I question the veracity of that. Is SPF failing to perform it's intended function if an unauthorized server is blocked from sending email with an envelope from a domain with an SPF record ending in "-all"? I'll defend that SPF is doing exactly what it was designed to do. I'll agree that some people may be surprised at the message being blocked. But that's a failure of the users understanding of SPF, and not a failure of SPF itself. It may be a bad configuration / SPF record, but again, that's not a failure of SPF to do what it was designed to do. (actually, even with +all, some sites will fail it - especially because of it, as +all is sign of either intentional sloppy spammer or incompetent postmaster, both likely leading to spam coming from that site). That is a receiving site applying local policy to override what the purported sending site has published. -- I don't care how many receiving sites apply that mentality. It is still contrary to what the purported sending site published. Any SPF, no matter how correctly configured, will lead to false positives in some cases (e.g. encoutering mailing list or .forward not using VERP/SRS). Is that /truly/ a false positive? Or did SPF function the way that it was designed and configured? I believe that it did. I recently helped someone set up SPF for a new domain. It was working fine for a few weeks until they tried to use a 3rd party service to send email on their behalf. I said that's new information and we need to update SPF to authorize the 3rd party to send email on their behalf. Did SPF fail? Nope. SPF did exactly what it was designed to do. The user changed what they wanted and needed to update their SPF record accordingly. SPF didn't fail. The user failed to make a change ahead of time. It is inherit in the SPF protocol (which is why DMARC checks both DKIM and SPF, in order to reduce, but not eliminate, false positives). DMARC is applying local policy and deciding to ignore the information that SPF successfully provided. Providing unfavorable information is not a failure to provide information. We are NOT living in ideal world where everybody implements every existing standard. Agreed. But we can strive for the ideal world. By doing so we will end up closer to the ideal world than where we started, even if we fall short thereof. Thus, even most correctly configured SPF will sometimes softfail/fail, when it should not. I question the veracity of "should not". Grant. . . .
Re: Ensuring SPF/DKIM for @gmail.com
On 7/26/23 1:44 PM, Marc wrote: so your ip does not generate a softfail or fail I assume that you mean so that your outbound SMTP server is actually authorized in some capacity and fall under "all". Is that correct? When you configure your spf your result is either pass, softfail or fail I think we can agree that a correctly configured spf results in a pass, don't you? I agree that known and authorized sources of email for my domain are authorized in the SPF record for my domain. I thought you were alluding to a particular value of "all". I did not understand your statement to be about servers being authorized or not. I've had a number of conversations with people of late that seem to dislike "-all" almost as much as others dislike "+all". Grant. . . .
Re: Ensuring SPF/DKIM for @gmail.com
On 27/07/2023 05:09, Matija Nalis wrote: Any SPF, no matter how correctly configured, will lead to false positives in some cases (e.g. encoutering mailing list B.S. mailing lists have been smart enough for over 20 years to rewrite sender and not appear as a basic forwarder - which are you are correct, however there are forwarding abilities to rewrite sender which avoids this, its been 15 years or more since I've used procmail which by default did not. If you are going to dry-reach to support an argument, please use modern facts and not 1990's. I was a *very* early adopter of SPF back in late 90's and have had zero issues in 20 years in using SPF (as expected as an early adopter, teething issues as with all software needed fine tuning in very early days) -- Regards, Noel Butler This Email, including attachments, may contain legally privileged information, therefore at all times remains confidential and subject to copyright protected under international law. You may not disseminate this message without the authors express written authority to do so. If you are not the intended recipient, please notify the sender then delete all copies of this message including attachments immediately. Confidentiality, copyright, and legal privilege are not waived or lost by reason of the mistaken delivery of this message.
RE: Ensuring SPF/DKIM for @gmail.com
> > > > > > What does "correctly setup SPF" mean to you? > > > > so your ip does not generate a softfail or fail > > Only way to make SPF never incorrectly fail/softwail is to use "+all", > but that kind of kills its point :-) +all is in pass https://datatracker.ietf.org/doc/html/rfc4408#page-8 > (actually, even with +all, some sites will fail it - especially > because of it, as +all is sign of either intentional sloppy spammer > or incompetent postmaster, both likely leading to spam coming from > that site). I am not even sure if I am able to differentiate on this level in my milter. > > > What makes your opinion better than someone else's opinion that > differs? > > > (I take it for granted that someone will have a differing > opinion.) > > > > When you configure your spf your result is either pass, softfail or > fail > > I think we can agree that a correctly configured spf results in a > pass, don't you? > > Well *I* don't. Sometimes, maybe even often, it does. But not always. > > Any SPF, no matter how correctly configured, will lead to false > positives in some cases (e.g. encoutering mailing list or .forward No not, the sender chooses this setup, so there are no false positives. The sender does not want your server to send email from their domain. The only reason I can think of, for allowing fail/softfail is if you do not know your own infrastructure wel enough. .forward should be set to forward with your own email address if spf is configured for external, or if it stays internal, spf should be skipped. > We are NOT living in ideal world where everybody implements every > existing standard. Thus, even most correctly configured SPF will > sometimes softfail/fail, when it should not. > This is just crap. I think 99% of the implemented spf checks are not following your reasoning. It is like you are telling your bank please I would like to use these 2 payment cards to spend from my bank account. And because it is not an ideal world, your bank will allow me to spend from your account?
Re: Ensuring SPF/DKIM for @gmail.com
On Wed, Jul 26, 2023 at 06:44:32PM +, Marc wrote: > > At the risk of starting a flame war... > > > > What does "correctly setup SPF" mean to you? > > so your ip does not generate a softfail or fail Only way to make SPF never incorrectly fail/softwail is to use "+all", but that kind of kills its point :-) (actually, even with +all, some sites will fail it - especially because of it, as +all is sign of either intentional sloppy spammer or incompetent postmaster, both likely leading to spam coming from that site). > > What makes your opinion better than someone else's opinion that differs? > > (I take it for granted that someone will have a differing opinion.) > > When you configure your spf your result is either pass, softfail or fail > I think we can agree that a correctly configured spf results in a pass, don't > you? Well *I* don't. Sometimes, maybe even often, it does. But not always. Any SPF, no matter how correctly configured, will lead to false positives in some cases (e.g. encoutering mailing list or .forward not using VERP/SRS). It is inherit in the SPF protocol (which is why DMARC checks both DKIM and SPF, in order to reduce, but not eliminate, false positives). We are NOT living in ideal world where everybody implements every existing standard. Thus, even most correctly configured SPF will sometimes softfail/fail, when it should not. Trying to pretend that the world is ideal is not really good idea; one might as well pretend that spam does not exist and save all that time wasted on implementing antispam measures :-) -- Opinions above are GNU-copylefted.
RE: Ensuring SPF/DKIM for @gmail.com
> At the risk of starting a flame war... > > What does "correctly setup SPF" mean to you? so your ip does not generate a softfail or fail > What makes your opinion better than someone else's opinion that differs? > (I take it for granted that someone will have a differing opinion.) When you configure your spf your result is either pass, softfail or fail I think we can agree that a correctly configured spf results in a pass, don't you? > I get your server your rules. But your server / your rules doesn't > translate to someone else's server any more than someone else's rules > translate to your server. > I agree
Re: Ensuring SPF/DKIM for @gmail.com
On 7/26/23 2:34 AM, Benny Pedersen wrote:
milters should not be spam scanners, spamassassin is better
{spamass-milter,milter-spamc} combined with SpamAssassin cause me to
question the veracity of that statement.
Milter implies doing the filtering during the SMTP transaction. I
consider the ability to reject messages that SpamAssassin declares as
(bad enough) spam at SMTP time to be a good thing.
maybe use bind9 rpz to change spf data for stupid domains, mostly
freemail domains in this category ? :)
Will you provide a description of what you would have an RPZ do?
I know that you can specify local data overriding the upstream DNS zone
data.
All of the local data would be locally and artificially baked. I guess
you could do some sort of query and create a fresh RPZ record with a
different "all" value. Or are you thinking something else?
This might be a use case for RPS to more dynamically alter the record.
But I've not messed with RPS much. Conversely I've done a lot with RPZ.
Grant. . . .
Re: Ensuring SPF/DKIM for @gmail.com
On 7/26/23 1:44 AM, Marc wrote: asking them to correctly setup spf is mostly enough. At the risk of starting a flame war... What does "correctly setup SPF" mean to you? What makes your opinion better than someone else's opinion that differs? (I take it for granted that someone will have a differing opinion.) I get your server your rules. But your server / your rules doesn't translate to someone else's server any more than someone else's rules translate to your server. Grant. . . .
Re: Ensuring SPF/DKIM for @gmail.com
On 26/07/2023 17:34, Benny Pedersen wrote: milters should not be spam scanners, spamassassin is better SA is perl, perl is faster and better resource nice than python garbage, but perl is still slow compared to C, that is why milters will win out everytime. milter-regex is also light and super speedy, it stops a lot of trash before postfix even accepts the message to give to SA Frankly google is just trash anyway, so anything that blocks gmail/google spam is a great idea. (have they stopped google groups from backscatter yet, probably not, they are too busy fscking over youtube) -- Regards, Noel Butler - Ensuring my "long sig" is even longer, just for Benny This Email, including attachments, may contain legally privileged information, therefore at all times remains confidential and subject to copyright protected under international law. You may not disseminate this message without the authors express written authority to do so. If you are not the intended recipient, please notify the sender then delete all copies of this message including attachments immediately. Confidentiality, copyright, and legal privilege are not waived or lost by reason of the mistaken delivery of this message. This I care not about mailing lists, although this is only a list/newsletter account its not my personal account Its one of my older formal but not personal addresses nbot really used in that lght now. thinking of other stuff to put in to annoy Benny but damn running outof time, time to go home its 5.50 pm
Re: Ensuring SPF/DKIM for @gmail.com
Marc skrev den 2023-07-26 08:44: blocklist_from *@gmail.com welcomelist_auth *@gmail.com makes it perfect :) if both dkim and spf is pass, it will get neutral scores I found this to be not sufficient (assuming the above pass is ~all). gmail has spf ~all. set softfail score to 100, solved So I have made an exception for the google network in milter and everything from the gmail / google that would fail an -all spf I reject. milters should not be spam scanners, spamassassin is better There is only a few legitimate domains that will be targetted by this, but asking them to correctly setup spf is mostly enough. maybe use bind9 rpz to change spf data for stupid domains, mostly freemail domains in this category ? :)
RE: Ensuring SPF/DKIM for @gmail.com
> > blocklist_from *@gmail.com > welcomelist_auth *@gmail.com > > makes it perfect :) > > if both dkim and spf is pass, it will get neutral scores > I found this to be not sufficient (assuming the above pass is ~all). gmail has spf ~all. So I have made an exception for the google network in milter and everything from the gmail / google that would fail an -all spf I reject. There is only a few legitimate domains that will be targetted by this, but asking them to correctly setup spf is mostly enough.
