Re: Penalty for no/bad SPF
On Wed, 24 Jan 2018, Bill Cole wrote: On 24 Jan 2018, at 17:20 (-0500), John Hardin wrote: On Wed, 24 Jan 2018, Dianne Skoll wrote: At this point, I would be willing to penalize sites with bad SPF records (syntactically invalid; more than one different SPF record attached to the same domain, etc.) Those people really deserve penalties because they've messed up. Does that include "+all" or authorizing more than a class-b space through any method, which I'd characterize as "malicious" rather than "messed up"? There are entities that still hold fast to their legacy Class A networks and expose them to some degree to the world. Those who have tried to change policy from inside such an organization might argue that a multiple-B SPF authorization is neither malicious nor messed up in itself, but rather merely an admission of a reality which i arguably messed up but not at all malicious. Somebody legitimately that big could be whitelisted (SPF Malice score = 0). -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- To be civilized is to restrain the ability to commit mayhem. To be incapable of committing mayhem is not the mark of the civilized, merely the domesticated.-- Trefor Thomas --- 3 days until Wolfgang Amadeus Mozart's 262nd Birthday
Re: Penalty for no/bad SPF
On 24 Jan 2018, at 17:20 (-0500), John Hardin wrote: On Wed, 24 Jan 2018, Dianne Skoll wrote: At this point, I would be willing to penalize sites with bad SPF records (syntactically invalid; more than one different SPF record attached to the same domain, etc.) Those people really deserve penalties because they've messed up. Does that include "+all" or authorizing more than a class-b space through any method, which I'd characterize as "malicious" rather than "messed up"? There are entities that still hold fast to their legacy Class A networks and expose them to some degree to the world. Those who have tried to change policy from inside such an organization might argue that a multiple-B SPF authorization is neither malicious nor messed up in itself, but rather merely an admission of a reality which i arguably messed up but not at all malicious. -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) Currently Seeking Steady Work: https://linkedin.com/in/billcole
Re: Penalty for no/bad SPF
On Wed, 24 Jan 2018, Dianne Skoll wrote: On Wed, 24 Jan 2018 14:20:57 -0800 (PST) John Hardin wrote: At this point, I would be willing to penalize sites with bad SPF records (syntactically invalid; more than one different SPF record attached to the same domain, etc.) Those people really deserve penalties because they've messed up. Does that include "+all" or authorizing more than a class-b space through any method, which I'd characterize as "malicious" rather than "messed up"? +all is malicious for sure. More than a Class-B might just be bad planning AKA Microsoft's outbound IP address list. I was thinking more the case where subrange assignments were used to avoid explicitly using "+all" as a way to avoid naive malice checks, e.g. doing "+ip4:0.0.0.0/1 +ip4:255.0.0.0/1". For reliability the threshold size might need to be larger than a class-B (that was off-the-cuff), or it might need some explicit whitelisting for broken-yet-legit domains (AKA msft). However, a malicious actor can use the "exists:" mechanism to simulate +all in a way that can't easily be proven by an SPF evaluator. :( I would like to see the exists: mechanism tossed. So we add a point for using "exists:" and for defining multiple subranges that add up to > class-B (or whatever threshold seems reasonable) and for any other constructs that are abused by spammers to define "SPF pass wherever my bots send from". It's sounding like we need "SPF cluebat" and "SPF malice" scoring plugins... :) -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- ...intellectuals have no interest in what _creates_ wealth, and what _inhibits_ the creation of wealth. They are very concerned about the _distribution_ of it, but they act as if wealth just exists somehow. It's like manna from heaven, it's only a question of how we split it up.-- Thomas Sowell --- 3 days until the 51st anniversary of the loss of Apollo 1
Re: Pretty good spoof of AmEx
On Thu, 25 Jan 2018 01:05:48 + RW wrote: > On Wed, 24 Jan 2018 03:07:48 -0500 > Rupert Gallagher wrote: > > > To: address matches Reply-To: address. > > > From: Rupert Gallagher > Reply-To: Rupert Gallagher Sorry I misread that.
Re: Pretty good spoof of AmEx
On Wed, 24 Jan 2018 03:07:48 -0500 Rupert Gallagher wrote: > To: address matches Reply-To: address. From: Rupert Gallagher Reply-To: Rupert Gallagher
Re: Penalty for no/bad SPF
On Wed, 24 Jan 2018 14:20:57 -0800 (PST) John Hardin wrote: > > At this point, I would be willing to penalize sites with bad SPF > > records (syntactically invalid; more than one different SPF record > > attached to the same domain, etc.) Those people really deserve > > penalties because they've messed up. > Does that include "+all" or authorizing more than a class-b space > through any method, which I'd characterize as "malicious" rather than > "messed up"? +all is malicious for sure. More than a Class-B might just be bad planning AKA Microsoft's outbound IP address list. However, a malicious actor can use the "exists:" mechanism to simulate +all in a way that can't easily be proven by an SPF evaluator. :( I would like to see the exists: mechanism tossed. Regards, Dianne.
Re: kam corpus
On 24 Jan 2018, at 04:08, Rupert Gallagher wrote: > Is this the "official" version of kam.cf? > > http://www.pccc.com/downloads/SpamAssassin/contrib/ > > The file is huge, and consists of ad-hoc rules against spammy keywords. Is less than 300K huge? That does remind me, though, does SpamAssassin automatically load *.cf in /usr/local/etc/mail/SpamAssassin or do extra cf files like KAM need to be added somewhere to be loaded? I seem to recall having to do something, but ti's been a long time since I did anything outside of local.cf -- ...but the senator, while insisting he was not intoxicated, could not explain his nudity.
Re: Penalty for no/bad SPF
On 2018-01-24 18:10, Bill Cole wrote: > 1. Mail with an envelope sender domain that has no SPF record is more > likely to be spam than the overall mail stream. > > 2. Mail whose envelope sender domain has a published SPF record which > repudiates the sending IP is more likely to be spam than the overall > mail stream. > > I don't see evidence that either of those are true now, that they have > ever been true, or that they are becoming closer to true over time. I am not taking sides in this dispute, but I'd like to point out that there is a phenomenon called "self-fulfilling prophesy". As in all affairs that involve human behavior and persuasion. -- Please don't Cc: me privately on mailing lists and Usenet, if you also post the followup to the list or newsgroup. To reply privately _only_ on Usenet, fetch the TXT record for the domain.
Re: Penalty for no/bad SPF
SPF is designed for authentication, not spam filtering. Using a crowbar as a hammer. We apply a small score mainly so we see the elements reported. If the "majors" are using in their hygiene stack, for evalation like you are, I haven't seen much evidence of that. Of course it's hard to test, since we don't have log access or often intelligible header diagnostics. But from years of blackbox practical experience working cases: Cust: My Spam...err bulkmail is going to Junk, can you hook me up w/ SPF! Tech : OK we hooked you up. Cust: My Spamerr bulkmail is still going to Junk Tech: OK now diagnosed your ACTUAL problem (URL, PTR, spammy content. etc). We need to do X, Y, and Z. Cust: Great that fixed it! I can't recall a case in the last 5 years where SPF, or DKIM for that matter, tipped anyone from Junk to not. I apply an increased score for RDNS_NONE. Because I think its an ABOMINATION that so many operations think it OK to skip DNS plumbing. But I do recognize it seems a hopeless battle, trying to clean up the internet. YMMV. From: Rupert Gallagher Sent: Wednesday, January 24, 2018 3:00:37 PM To: David Jones; 'users@spamassassin.apache.org' Subject: Re: Penalty for no/bad SPF If all those smarties who speak against spf would simply shut-up and implement it correctly, with dkim and dmarc, and read the dmarc reports, then they would know how valuable it is. On raising the score: done, long ago, happy with the outcome, and strongly recommend it.
Re: Penalty for no/bad SPF
On 24 Jan 2018, at 14:59 (-0500), David Jones wrote: On 01/24/2018 01:33 PM, Bill Cole wrote: On 24 Jan 2018, at 9:12, David Jones wrote: What does everyone think about slowly increasing the score for SPF_NONE and SPF_FAIL over time in the SA rulesets to force the awareness and importance of proper SPF? -1 In every real mailstream I've worked with in the lifetime of SPF, lack of SPF has *always* had a correlation with ham, not spam. I am not suggesting that SPF_PASS = ham and SPF_FAIL = spam. I did not think or say that you were. You are proposing that SA as a project should *act as if* these correlations exist and are steadily becoming stronger: 1. Mail with an envelope sender domain that has no SPF record is more likely to be spam than the overall mail stream. 2. Mail whose envelope sender domain has a published SPF record which repudiates the sending IP is more likely to be spam than the overall mail stream. I don't see evidence that either of those are true now, that they have ever been true, or that they are becoming closer to true over time. SPF hard failures are a more complicated case because the sort of spam that hits SPF_FAIL tends to come from IPs that show up in good DNSBLs within a few minutes, making it hard for a site using DNSBLs to know how much of it there is. With that caveat, I see more ham hitting SPF_FAIL than I do spam where SPF_FAIL (which I have locally nailed at 2.0) is a decisive factor. Most SPF_FAIL spam scores well into double digits here. I am proposing that if SPF were more accurately deployed then SPF_FAIL would be worth something. That is a truism but what you are proposing is that SA should behave as if SPF deployment is becoming broader and more accurate steadily over time, as a sort of sympathetic magic to drive that improvement. I am saying that history implies that this would not work as hoped. Also, it is a bad precedent for project policy.Slowly rising SA scores don't exert any pressure on a sender with broken SPF until they actually cause mail to be dropped or rejected. We should not be intentionally increasing false positives no matter how noble the end goal. Individual sites obviously can make that choice on their own, now. For example, on my own personal system I would very likely see some FPs due to my unjustifiably high fixed score for SPF_FAIL, were it not for other bespoke tweaks which make actually effective FPs extremely unlikely. We could whitelist_auth more trusted senders and then be able to turn up the scores for the rest of the mail flow. If the huge SA community around the world were to help push SPF adoption and accurate deployments, then we could move on to DKIM too. Right now, the best option we have is to get DMARC properly deployed as much as possible where p=reject actually rejects the message unlike SPF_FAIL that we can't trust. SA is not the vehicle for such pressure. What has been driving careful DMARC deployment is not the long tail of sites with <10k users using SA, it is the handful of behemoth providers plus some key commercial and government systems that honor p=reject. This was also the strongest driver for SPF adoption, until those behemoths recognized how flawed SPF & DKIM alone were and came up with DMARC as a "solution" with the unsurprising side effect of damaging traditional discussion mailing lists. -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) Currently Seeking Steady Work: https://linkedin.com/in/billcole
Re: Penalty for no/bad SPF
If all those smarties who speak against spf would simply shut-up and implement it correctly, with dkim and dmarc, and read the dmarc reports, then they would know how valuable it is. On raising the score: done, long ago, happy with the outcome, and strongly recommend it.
Re: Penalty for no/bad SPF
On Wed, 2018-01-24 at 14:24 -0800, John Hardin wrote: > I think he was referring to MTA-side forwarding, not forwarding an > email you received (which forward comes *from you*). > I was wondering if this could be related to Joseph's comment that "DMARC is destroying forwarding and mailing lists" as some sort of circumvention, thats's all. > > and the more so because this is not an optional feature. > > *that* is unjustified. > Agreed: I feel a Bugzilla request coming on. Martin
Re: Penalty for no/bad SPF
On 01/24/2018 04:00 PM, Vincent Fox wrote: so there's this argument that goes: "well we won't really see the benefits until it's FULLY and RIGIDLY implemented." However, look at all the major providers with messed up records and neutral or soft fail. They should have the most resources to accomplish this and the most incentives to list all their netblocks and set to hard fail. Google is soft fail. Hotmail is soft fail. (etc etc ad nauseum) I rest my case. There is nothing wrong with stopping a soft fail if that is what they want to do. In fact, most people should stop at soft fail unless they really know what they are doing or they are a major brand with a high risk spoofing. People are blindly following Microsoft's DNS entries for Office 365 setup with "-all" when they don't know what they are doing. Microsoft should be telling people to do "~all" in their setup instructions. Then Microsoft should be checking their customer's SPF records for them and showing them when it's broken in the the Admin Center. 1. We need SPF_FAIL hits to mean something and they don't. 2. We can use subdomains with SPF_PASS to safelist trusted senders that are targets of spoofing. After 14+ years we are still having this ridiculous argument about how in 14 MORE years when we finally fully implement this flawed technology, it'll do something useful. Meanwhile i see it as being more risk than benefit. With a big force like SA or Google, we could do this in a couple of years slowly and easily then start doing the same for DKIM. Frankly I'd rather these manhours be used on having correct A & PTR records, which seems to be beyond the pale for some bulkmail vendors. We could do the same thing for RDNS_NONE hits. Good idea. -- David Jones
Re: Penalty for no/bad SPF
On Wed, 24 Jan 2018, Martin Gregorie wrote: On Wed, 2018-01-24 at 16:45 -0500, Joseph Brennan wrote: DMARC is not a standard according to RFC 7489, "Status of This Memo". It's just informational, for those who want to play the game. DMARC is destroying forwarding and mailing lists, Could this be why recent releases of the Evolution MUA now send forwarded messages as attachments appended to a new message? I think he was referring to MTA-side forwarding, not forwarding an email you received (which forward comes *from you*). Its annoying, particularly if you're intending to intersperse comments in the original message as part of a detailed discussion, Reply? and the more so because this is not an optional feature. *that* is unjustified. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Vista is at best mildly annoying and at worst makes you want to rush to Redmond, Wash. and rip somebody's liver out. -- Forbes --- 3 days until Wolfgang Amadeus Mozart's 262nd Birthday
Re: Penalty for no/bad SPF
On Wed, 24 Jan 2018, Dianne Skoll wrote: At this point, I would be willing to penalize sites with bad SPF records (syntactically invalid; more than one different SPF record attached to the same domain, etc.) Those people really deserve penalties because they've messed up. Does that include "+all" or authorizing more than a class-b space through any method, which I'd characterize as "malicious" rather than "messed up"? -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Vista is at best mildly annoying and at worst makes you want to rush to Redmond, Wash. and rip somebody's liver out. -- Forbes --- 3 days until Wolfgang Amadeus Mozart's 262nd Birthday
Re: Penalty for no/bad SPF
On Wed, 2018-01-24 at 16:45 -0500, Joseph Brennan wrote: > DMARC is not a standard according to RFC 7489, "Status of This Memo". > It's just informational, for those who want to play the game. DMARC > is destroying forwarding and mailing lists, > Could this be why recent releases of the Evolution MUA now send forwarded messages as attachments appended to a new message? Its annoying, particularly if you're intending to intersperse comments in the original message as part of a detailed discussion, and the more so because this is not an optional feature. Apologies if this is a bit OTT, but I don't understand why anybody would think its a good way to handle forwarded messages. Martin
Re: Penalty for no/bad SPF
On 01/24/2018 03:45 PM, Joseph Brennan wrote: David Jones wrote: SA could be the large force that helps improve the mail standards like DMARC -- SPF + DKIM with a little extra on top. DMARC is not a standard according to RFC 7489, "Status of This Memo". It's just informational, for those who want to play the game. DMARC is destroying forwarding and mailing lists, and I'm sorry to see the elephants in the email room implementing it-- though Gmail still does not always reject based on DMARC reject, as if they use that plus some internal system to make the call. The New York Times nytimes.com has a SPF record with too many DNS lookups. Are you willing to block that? That one amazes me since SPF is the simplest of these ventures to implement correctly, and since the Times's frequent mailings of news updates evidently are not affected enough by SPF fail for the Times to go fix it. Joseph Brennan Columbia University IT The key point here is the bulk nytimes.com email that is system generated, i.e. not humans with real mailboxes that could be compromised, is from subdomains so this entry would be safe since they do have good SPF records on subdomains: whitelist_auth *@*.nytimes.com In fact, I have put this in the 60_whitelist_auth.cf: def_whitelist_auth *@*.nytimes.com ... so bulk emails from them should be scoring pretty low for everyone running sa-update. This should prove my main point of this thread. SA actually allows for more than 10 DNS lookups so it's more forgiving than the actual spec and would likely hit SPF_PASS still as long as the emails came from a covered source. I have a recent email from a human-looking email address @nytimes.com that SA shows as SPF_PASS from a Google mail server. -- David Jones
Re: Penalty for no/bad SPF
so there's this argument that goes: "well we won't really see the benefits until it's FULLY and RIGIDLY implemented." However, look at all the major providers with messed up records and neutral or soft fail. They should have the most resources to accomplish this and the most incentives to list all their netblocks and set to hard fail. Google is soft fail. Hotmail is soft fail. (etc etc ad nauseum) I rest my case. After 14+ years we are still having this ridiculous argument about how in 14 MORE years when we finally fully implement this flawed technology, it'll do something useful. Meanwhile i see it as being more risk than benefit. Frankly I'd rather these manhours be used on having correct A & PTR records, which seems to be beyond the pale for some bulkmail vendors. From: David Jones Sent: Wednesday, January 24, 2018 12:12:56 PM To: users@spamassassin.apache.org Subject: Re: Penalty for no/bad SPF On 01/24/2018 01:58 PM, Vincent Fox wrote: > I'd rather not think about the manhours I've wasted this year on SPF. > > > The guy at Evotec.com, among others, who thinks rejecting > > for SOFTFAIL is a perfectly valid anti-spoofing strategy and > > doesn't blink when pointed to RFC 4408 sec 2.5.5. > > > Vendors who's first response is: > > "Our LEGIT spamerrr bulkmail is ending in your Junk. Response > > #1 in our binder is you MUST list us in your SPF record." > > Dig, dig, dig maillogs. All emails using Envelope From properly > > so SPF is a waste of everyone's time. > The Internet is very slow to change. It takes a large force like Google to improve things slowly over time. They are doing good work in the TLS and browser encryption area. SA could be the large force that helps improve the mail standards like DMARC -- SPF + DKIM with a little extra on top. > > Records we included to ours, where the vendor makes a typo in > > THEIR SPF record on a Friday night. Or decides to add 9 sub-includes. > > Either way our record suddenly returning PERMERROR and we > > have to get someone in, and boot vendor off the island on a Sunday. > I have a script that checks all of our customer's SPF records for syntax problems and too many DNS lookups based on pyspf just like http://www.kitterman.com/spf/validate.html does so I can correct it or notify them immediately. > > Endless hours explaining to campus clients, what SPF is and why > If SA all around the world says the same thing you are telling them then they will have to listen and fix their problem or remove their SPF record which is better than having an incorrect one. > it is not a good primary strategy to solve Junk mail issues > > The only good thing I have to say about SPF, is it seems to > > be a permanent employment program for people who are > > otherwise useless. > -- David Jones
Re: Penalty for no/bad SPF
David Jones wrote: SA could be the large force that helps improve the mail standards like DMARC -- SPF + DKIM with a little extra on top. DMARC is not a standard according to RFC 7489, "Status of This Memo". It's just informational, for those who want to play the game. DMARC is destroying forwarding and mailing lists, and I'm sorry to see the elephants in the email room implementing it-- though Gmail still does not always reject based on DMARC reject, as if they use that plus some internal system to make the call. The New York Times nytimes.com has a SPF record with too many DNS lookups. Are you willing to block that? That one amazes me since SPF is the simplest of these ventures to implement correctly, and since the Times's frequent mailings of news updates evidently are not affected enough by SPF fail for the Times to go fix it. Joseph Brennan Columbia University IT
Re: Pretty good spoof of AmEx
On 23 Jan 2018, at 21:26 (-0500), David Jones wrote: We could setup a 60_blacklist_from.cf file in the SA ruleset for definite bad domains but that's probably not the best place to maintain that. It really should be in major DBLs that SA already knows to check. +1 Unsurprisingly, running a derogatory reputation service is a toxic quagmire that the Apache SpamAssassin Project should not dive into. I have nothing but awe and respect for Steve Linford & his team at Spamhaus, precisely because they have figured out how to survive while publicly saying blatantly unflattering things about many people who have little respect for law or basic human decency. Doing that requires human and technical infrastructure that I would not expect the ASF or independent benefactors to be able or willing to provide. IMHO it is risky enough that we essentially bless a subset of a restricted class of senders via the "default whitelist" subsystem and we don't need the headache of distributing a list of "bad guys" that would need constant diligent maintenance to avoid being de facto defamatory. After all, domain registrations do expire and people do blindly re-register "burner" domains that spammers have had their fill of and let expire. -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) Currently Seeking Steady Work: https://linkedin.com/in/billcole
Re: Penalty for no/bad SPF
At this point, I would be willing to penalize sites with bad SPF records (syntactically invalid; more than one different SPF record attached to the same domain, etc.) Those people really deserve penalties because they've messed up. I would not be willing to penalize sites with *no* SPF at all just yet. Maybe eventually. Regards, Dianne.
Re: Penalty for no/bad SPF
On 01/24/2018 01:58 PM, Vincent Fox wrote: I'd rather not think about the manhours I've wasted this year on SPF. The guy at Evotec.com, among others, who thinks rejecting for SOFTFAIL is a perfectly valid anti-spoofing strategy and doesn't blink when pointed to RFC 4408 sec 2.5.5. Vendors who's first response is: "Our LEGIT spamerrr bulkmail is ending in your Junk. Response #1 in our binder is you MUST list us in your SPF record." Dig, dig, dig maillogs. All emails using Envelope From properly so SPF is a waste of everyone's time. The Internet is very slow to change. It takes a large force like Google to improve things slowly over time. They are doing good work in the TLS and browser encryption area. SA could be the large force that helps improve the mail standards like DMARC -- SPF + DKIM with a little extra on top. Records we included to ours, where the vendor makes a typo in THEIR SPF record on a Friday night. Or decides to add 9 sub-includes. Either way our record suddenly returning PERMERROR and we have to get someone in, and boot vendor off the island on a Sunday. I have a script that checks all of our customer's SPF records for syntax problems and too many DNS lookups based on pyspf just like http://www.kitterman.com/spf/validate.html does so I can correct it or notify them immediately. Endless hours explaining to campus clients, what SPF is and why If SA all around the world says the same thing you are telling them then they will have to listen and fix their problem or remove their SPF record which is better than having an incorrect one. it is not a good primary strategy to solve Junk mail issues The only good thing I have to say about SPF, is it seems to be a permanent employment program for people who are otherwise useless. -- David Jones
Re: Penalty for no/bad SPF
I'd rather not think about the manhours I've wasted this year on SPF. The guy at Evotec.com, among others, who thinks rejecting for SOFTFAIL is a perfectly valid anti-spoofing strategy and doesn't blink when pointed to RFC 4408 sec 2.5.5. Vendors who's first response is: "Our LEGIT spamerrr bulkmail is ending in your Junk. Response #1 in our binder is you MUST list us in your SPF record." Dig, dig, dig maillogs. All emails using Envelope From properly so SPF is a waste of everyone's time. Records we included to ours, where the vendor makes a typo in THEIR SPF record on a Friday night. Or decides to add 9 sub-includes. Either way our record suddenly returning PERMERROR and we have to get someone in, and boot vendor off the island on a Sunday. Endless hours explaining to campus clients, what SPF is and why it is not a good primary strategy to solve Junk mail issues. The only good thing I have to say about SPF, is it seems to be a permanent employment program for people who are otherwise useless. From: Martin Gregorie Sent: Wednesday, January 24, 2018 11:32:27 AM To: users@spamassassin.apache.org Subject: Re: Penalty for no/bad SPF On Wed, 2018-01-24 at 19:01 +, Vincent Fox wrote: > SPF is a zombie legacy that someone should shoot in > the head. > SPF is still good for what I've always thought was its main use: detecting spam delivered by backscatter. Given that its dirt cheap to implement, and easy too verify now that there are good checking tools there's no reason or necessity to kill it. > Maybe then we could design something that is useful for what we all > desire, which is properly authenticating senders. > ... provided that you can persuade all legit senders to buy into using it while preventing all spammers and bots from hijacking it. Martin
Re: Penalty for no/bad SPF
On 01/24/2018 01:33 PM, Bill Cole wrote: On 24 Jan 2018, at 9:12, David Jones wrote: What does everyone think about slowly increasing the score for SPF_NONE and SPF_FAIL over time in the SA rulesets to force the awareness and importance of proper SPF? -1 In every real mailstream I've worked with in the lifetime of SPF, lack of SPF has *always* had a correlation with ham, not spam. I am not suggesting that SPF_PASS = ham and SPF_FAIL = spam. SPF hard failures are a more complicated case because the sort of spam that hits SPF_FAIL tends to come from IPs that show up in good DNSBLs within a few minutes, making it hard for a site using DNSBLs to know how much of it there is. With that caveat, I see more ham hitting SPF_FAIL than I do spam where SPF_FAIL (which I have locally nailed at 2.0) is a decisive factor. Most SPF_FAIL spam scores well into double digits here. I am proposing that if SPF were more accurately deployed then SPF_FAIL would be worth something. We could whitelist_auth more trusted senders and then be able to turn up the scores for the rest of the mail flow. If the huge SA community around the world were to help push SPF adoption and accurate deployments, then we could move on to DKIM too. Right now, the best option we have is to get DMARC properly deployed as much as possible where p=reject actually rejects the message unlike SPF_FAIL that we can't trust. -- David Jones
Re: Penalty for no/bad SPF
On 01/24/2018 01:01 PM, Vincent Fox wrote: SPF is designed for whitelisting, not blacklist. Not true. We are supposed to reject email that doesn't pass SPF checks if the domain has told us to with "-all". That is a form of blacklisting controlled by the domain itself to protect it's own brand/reputation from spoofing. If you don't know what you are doing and create an SPF record with "-all", then that's your own fault. Google has started putting many of these emails with SPF_FAIL in their Spam folder which puts more importance to get your SPF record correct and complete. Remember when "shields" appeared in mail clients, and how fast that feature disappeared? Far too many people clicking on phish that seemed "authentic". With the explosion of cheap domains and registrars, there's really no snowshoe Black Hat operation that can't comply. Compliance is 99.% in every phish I've investigated last year, the outlier I can recall was a simple typo in 1 server in the sender's network. SPF is authorization not authentication. There is a difference and complete security requires both. You use SPF to tell the Internet authorized mail servers for your domain. That doesn't have a direct tie to spam but it does allow a level of whitelisting (I like to call it safelisting) for senders you trust. SPF is a zombie legacy that someone should shoot in the head. Maybe then we could design something that is useful for what we all desire, which is properly authenticating senders. If keep in mind that SPF is authorization and DKIM is authentication, you really need both. When a sender with DMARC p=reject aligns perfectly in both SPF and DKIM then all you know is that email is legit and not spoofed. It doesn't say anything about spam or ham in the content. If you whitelist trusted senders then you segment them out of the way which allows fine tuning on the rest of the mail flow. *From:* David Jones *Sent:* Wednesday, January 24, 2018 6:12:19 AM *To:* 'users@spamassassin.apache.org' *Subject:* Penalty for no/bad SPF Google Chrome and other browsers have been slowly penalizing sites not using encryption to the point that soon they will be alerting users of plain HTTP sites. This along with letsencrypt.org has been moving the HTTPS bar forward to improve web security and privacy. I think it's time for the SA community to help move the bar forward with SPF. The problem is many sysadmins that don't understand SPF have been implementing SPF incorrectly (thank you Microsoft Office 365) and incompletely without understanding they are shooting themselves in the foot. I decided about a month ago to start sending feedback emails to senders with SPF PERMERR and SPF FAIL in an attempt to help them help themselves improve _their_ mail delivery. If you setup your SPF record like Microsoft recommends with a "-all" and it's not completely covering all legit sources of email, it's completely useless for any MTAs and mail filters to take SPF_FAIL hits seriously. We should have rejected the email per that sending domain's own wishes but we all know they didn't intend for us to really block it so what good is it? What does everyone think about slowly increasing the score for SPF_NONE and SPF_FAIL over time in the SA rulesets to force the awareness and importance of proper SPF? This may need to have an official announcement of what the plans/timelines would be so we could get the word out. These days with DMARC reporting, it's not impossible to figure out a good SPF record like it was 10 years ago. The real problem with SMTP in general is there is no reliable way to get feedback to mail admins without sending confusing technical emails to regular users. -- David Jones -- David Jones
Re: Penalty for no/bad SPF
On 24 Jan 2018, at 9:12, David Jones wrote: What does everyone think about slowly increasing the score for SPF_NONE and SPF_FAIL over time in the SA rulesets to force the awareness and importance of proper SPF? -1 In every real mailstream I've worked with in the lifetime of SPF, lack of SPF has *always* had a correlation with ham, not spam. SPF hard failures are a more complicated case because the sort of spam that hits SPF_FAIL tends to come from IPs that show up in good DNSBLs within a few minutes, making it hard for a site using DNSBLs to know how much of it there is. With that caveat, I see more ham hitting SPF_FAIL than I do spam where SPF_FAIL (which I have locally nailed at 2.0) is a decisive factor. Most SPF_FAIL spam scores well into double digits here.
Re: Penalty for no/bad SPF
On Wed, 2018-01-24 at 19:01 +, Vincent Fox wrote: > SPF is a zombie legacy that someone should shoot in > the head. > SPF is still good for what I've always thought was its main use: detecting spam delivered by backscatter. Given that its dirt cheap to implement, and easy too verify now that there are good checking tools there's no reason or necessity to kill it. > Maybe then we could design something that is useful for what we all > desire, which is properly authenticating senders. > ... provided that you can persuade all legit senders to buy into using it while preventing all spammers and bots from hijacking it. Martin
Re: Penalty for no/bad SPF
On Wed, 24 Jan 2018 19:01:28 + Vincent Fox wrote: > SPF is a zombie legacy that someone should shoot in > the head. +1 > Maybe then we could design something that > is useful for what we all desire, which is properly > authenticating senders. We cannot authenticate senders and keep SMTP as useful as it is now. The whole beauty of email is that some stranger can contact you out of the blue. And that mailing lists can send mail on your behalf. We can work on people not being able to spoof "well-known" senders; for that, I'd much prefer DKIM+DMARC over SPF. SPF was a mistake and we should all admit that. Regards, Dianne.
Re: kam corpus
We had three spam messages in about 8 months? I lost count. Our clients are so used to have a clean inbox that they spot a spam like the proverbial white fly. Sent from ProtonMail Mobile On Wed, Jan 24, 2018 at 13:34, Kevin A. McGrail wrote: > On 1/24/2018 6:08 AM, Rupert Gallagher wrote: > >> Is this the "official" version of kam.cf? >> >> http://www.pccc.com/downloads/SpamAssassin/contrib/ > > Yes. Are there unofficial versions? > >> The file is huge, and consists of ad-hoc rules against spammy keywords. >> >> We use a completely different approach, resulting in few general rules and a >> short whitelist. We hardly see any kam-esque spam, but we are wise enough to >> verify. Is there an open corpus of kam-spam that we can process? > > Sorry, no, we do not provide a spam or ham corpora for verification. I can > tell you that we get about 2 problem reports a week average with 100's of > millions of mailboxes using our cf.
Re: Penalty for no/bad SPF
SPF is designed for whitelisting, not blacklist. Remember when "shields" appeared in mail clients, and how fast that feature disappeared? Far too many people clicking on phish that seemed "authentic". With the explosion of cheap domains and registrars, there's really no snowshoe Black Hat operation that can't comply. Compliance is 99.% in every phish I've investigated last year, the outlier I can recall was a simple typo in 1 server in the sender's network. SPF is a zombie legacy that someone should shoot in the head. Maybe then we could design something that is useful for what we all desire, which is properly authenticating senders. From: David Jones Sent: Wednesday, January 24, 2018 6:12:19 AM To: 'users@spamassassin.apache.org' Subject: Penalty for no/bad SPF Google Chrome and other browsers have been slowly penalizing sites not using encryption to the point that soon they will be alerting users of plain HTTP sites. This along with letsencrypt.org has been moving the HTTPS bar forward to improve web security and privacy. I think it's time for the SA community to help move the bar forward with SPF. The problem is many sysadmins that don't understand SPF have been implementing SPF incorrectly (thank you Microsoft Office 365) and incompletely without understanding they are shooting themselves in the foot. I decided about a month ago to start sending feedback emails to senders with SPF PERMERR and SPF FAIL in an attempt to help them help themselves improve _their_ mail delivery. If you setup your SPF record like Microsoft recommends with a "-all" and it's not completely covering all legit sources of email, it's completely useless for any MTAs and mail filters to take SPF_FAIL hits seriously. We should have rejected the email per that sending domain's own wishes but we all know they didn't intend for us to really block it so what good is it? What does everyone think about slowly increasing the score for SPF_NONE and SPF_FAIL over time in the SA rulesets to force the awareness and importance of proper SPF? This may need to have an official announcement of what the plans/timelines would be so we could get the word out. These days with DMARC reporting, it's not impossible to figure out a good SPF record like it was 10 years ago. The real problem with SMTP in general is there is no reliable way to get feedback to mail admins without sending confusing technical emails to regular users. -- David Jones
Re: Penalty for no/bad SPF
On Wed, Jan 24, 2018 at 08:12:19AM -0600, David Jones wrote: > Google Chrome and other browsers have been slowly penalizing sites not > using encryption to the point that soon they will be alerting users of > plain HTTP sites. This along with letsencrypt.org has been moving the > HTTPS bar forward to improve web security and privacy. > > I think it's time for the SA community to help move the bar forward with > SPF. The problem is many sysadmins that don't understand SPF have been > implementing SPF incorrectly (thank you Microsoft Office 365) and > incompletely without understanding they are shooting themselves in the foot. > > I decided about a month ago to start sending feedback emails to senders > with SPF PERMERR and SPF FAIL in an attempt to help them help themselves > improve _their_ mail delivery. If you setup your SPF record like > Microsoft recommends with a "-all" and it's not completely covering all > legit sources of email, it's completely useless for any MTAs and mail > filters to take SPF_FAIL hits seriously. We should have rejected the > email per that sending domain's own wishes but we all know they didn't > intend for us to really block it so what good is it? > > What does everyone think about slowly increasing the score for SPF_NONE > and SPF_FAIL over time in the SA rulesets to force the awareness and > importance of proper SPF? This may need to have an official > announcement of what the plans/timelines would be so we could get the > word out. > it sounds like a plan, I am all for that. Giovanni > These days with DMARC reporting, it's not impossible to figure out a > good SPF record like it was 10 years ago. The real problem with SMTP in > general is there is no reliable way to get feedback to mail admins > without sending confusing technical emails to regular users. > > -- > David Jones signature.asc Description: PGP signature
Penalty for no/bad SPF
Google Chrome and other browsers have been slowly penalizing sites not using encryption to the point that soon they will be alerting users of plain HTTP sites. This along with letsencrypt.org has been moving the HTTPS bar forward to improve web security and privacy. I think it's time for the SA community to help move the bar forward with SPF. The problem is many sysadmins that don't understand SPF have been implementing SPF incorrectly (thank you Microsoft Office 365) and incompletely without understanding they are shooting themselves in the foot. I decided about a month ago to start sending feedback emails to senders with SPF PERMERR and SPF FAIL in an attempt to help them help themselves improve _their_ mail delivery. If you setup your SPF record like Microsoft recommends with a "-all" and it's not completely covering all legit sources of email, it's completely useless for any MTAs and mail filters to take SPF_FAIL hits seriously. We should have rejected the email per that sending domain's own wishes but we all know they didn't intend for us to really block it so what good is it? What does everyone think about slowly increasing the score for SPF_NONE and SPF_FAIL over time in the SA rulesets to force the awareness and importance of proper SPF? This may need to have an official announcement of what the plans/timelines would be so we could get the word out. These days with DMARC reporting, it's not impossible to figure out a good SPF record like it was 10 years ago. The real problem with SMTP in general is there is no reliable way to get feedback to mail admins without sending confusing technical emails to regular users. -- David Jones
Re: kam corpus
On 1/24/2018 6:08 AM, Rupert Gallagher wrote: Is this the "official" version of kam.cf? http://www.pccc.com/downloads/SpamAssassin/contrib/ Yes. Are there unofficial versions? The file is huge, and consists of ad-hoc rules against spammy keywords. We use a completely different approach, resulting in few general rules and a short whitelist. We hardly see any kam-esque spam, but we are wise enough to verify. Is there an open corpus of kam-spam that we can process? Sorry, no, we do not provide a spam or ham corpora for verification. I can tell you that we get about 2 problem reports a week average with 100's of millions of mailboxes using our cf.
kam corpus
Is this the "official" version of kam.cf? http://www.pccc.com/downloads/SpamAssassin/contrib/ The file is huge, and consists of ad-hoc rules against spammy keywords. We use a completely different approach, resulting in few general rules and a short whitelist. We hardly see any kam-esque spam, but we are wise enough to verify. Is there an open corpus of kam-spam that we can process?
Re: Pretty good spoof of AmEx
To: address matches Reply-To: address. Sent from ProtonMail Mobile
