Re: [mailop] SendGrid is deleting your mail
Dňa 25. júna 2023 15:39:44 UTC používateľ Andy Smith via mailop napísal: >I think that you are perhaps only considering this from the >perspective of a sender. When it comes to choosing text for SMTP >responses there are many different types of person involved and many >of them will not be thinking about "what will large senders think >about this text?" Even if maybe they should have. Why we should have to discriminate senders by size? AFAIK, RFC says that text after code is not significant. My understanding of that is, that response has two parts, the one is code, the other is text, where that code is intended for machines and text for senders. The senders are not big nor small, they are (mostly) people. If someone use automated email system, without people as sender, it is he/she freedom, but cannot expect that whole world will care about this. Not repeat delivery due word "blacklist" means for me -- we know that we send craps and thus we can end in blacklist, but we don't bother with solving that, as we expect it. regards -- Slavko https://www.slavino.sk/ ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
Yes, me being only on the senders' side for now is driving some of the bias. I'll work on that. I agree with the rest of your mail, except for "actionable" being equal to "how to deliver this message in future" - I would consider *"we'll never accept mail from you"* to be also very actionable with the action being "don't mail". Otherwise I think we're on the same page, thank you. On Sun, 25 Jun 2023 at 16:43, Andy Smith via mailop wrote: > Hi Dmytro, > > On Sun, Jun 25, 2023 at 02:28:33PM +0100, Dmytro Homoniuk via mailop wrote: > > 450 4.3.2 Local problem - couldn't query foobar blacklist > > > > I do think this very hypothetical example is a bit of an outlier. It's > > providing non-actionable information to the sending system: it should > read > > this and ... erm... reach out to you and tell you you may have your > > blacklist API malfunctioning? > > I think that you are perhaps only considering this from the > perspective of a sender. When it comes to choosing text for SMTP > responses there are many different types of person involved and many > of them will not be thinking about "what will large senders think > about this text?" Even if maybe they should have. > > Obviously you have the software developers of the mail servers > involved. Then you have the administrators of the systems who are > doing what they think is operationally best. Speaking as an > operator, I would previously have not hesitated to include some text > that I, my colleagues, my monitoring systems and any person watching > a port 25 conversation, might find valuable. Yes, it's often better > put in the logs and not sent out on the wire back to a client, but > that was previously not a strong concern for me and I think it's > also very likely not a big concern for many others making these > decisions. > > So should we be putting out best practices documents for software > authors and systems administrators that say: > > When considering responses: > > - Only provide information that is actionable advice for the > client. > > - Err on the side of being terse. Be more verbose in logs only. > > I mean, that sort of advice seems reasonable anyway, but here we are > talking about it not just being reasonable, but in fact the > consequences may be, "or your user's mail may be silently deleted." > > > As as sender I would be very satisfied with *"450 4.3.2 Local problem - > > retry later"* - this way you'd tell me the deferral is not exactly my > > fault, it's you and I'm not expected to figure out the issue on my own. > And > > yes, if it persists - I'd be reaching out to ask about it, so if there is > > anything I can do - I would want it in the response. > > Thing is, many of us thought we were operating in a world where the > text of a response was meant to explain what happened to anyone > interested, not SOLELY for telling senders what they need to do to > get this message delivered in future. What we considered we had at > our disposal for THAT was either 4xx or 5xx and that's it. > > Isn't this an example of senders arguing that the SMTP response is > just for them, at the expense of everyone else who might have got > some use out of it? > > The rest of your message tries to show an equivalence between > Michael deliberately temporarily rejecting an email (as opposed to > because of some unexpected problem) and SendGrid deciding to discard > it without any retries. I've mentioned before that I'm not really > interested in debating that one because my main concern here is the > unintended consequences of this practice. > > Thanks, > Andy > > -- > https://bitfolk.com/ -- No-nonsense VPS hosting > ___ > mailop mailing list > mailop@mailop.org > https://list.mailop.org/listinfo/mailop > ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
Hi Dmytro, On Sun, Jun 25, 2023 at 02:28:33PM +0100, Dmytro Homoniuk via mailop wrote: > 450 4.3.2 Local problem - couldn't query foobar blacklist > > I do think this very hypothetical example is a bit of an outlier. It's > providing non-actionable information to the sending system: it should read > this and ... erm... reach out to you and tell you you may have your > blacklist API malfunctioning? I think that you are perhaps only considering this from the perspective of a sender. When it comes to choosing text for SMTP responses there are many different types of person involved and many of them will not be thinking about "what will large senders think about this text?" Even if maybe they should have. Obviously you have the software developers of the mail servers involved. Then you have the administrators of the systems who are doing what they think is operationally best. Speaking as an operator, I would previously have not hesitated to include some text that I, my colleagues, my monitoring systems and any person watching a port 25 conversation, might find valuable. Yes, it's often better put in the logs and not sent out on the wire back to a client, but that was previously not a strong concern for me and I think it's also very likely not a big concern for many others making these decisions. So should we be putting out best practices documents for software authors and systems administrators that say: When considering responses: - Only provide information that is actionable advice for the client. - Err on the side of being terse. Be more verbose in logs only. I mean, that sort of advice seems reasonable anyway, but here we are talking about it not just being reasonable, but in fact the consequences may be, "or your user's mail may be silently deleted." > As as sender I would be very satisfied with *"450 4.3.2 Local problem - > retry later"* - this way you'd tell me the deferral is not exactly my > fault, it's you and I'm not expected to figure out the issue on my own. And > yes, if it persists - I'd be reaching out to ask about it, so if there is > anything I can do - I would want it in the response. Thing is, many of us thought we were operating in a world where the text of a response was meant to explain what happened to anyone interested, not SOLELY for telling senders what they need to do to get this message delivered in future. What we considered we had at our disposal for THAT was either 4xx or 5xx and that's it. Isn't this an example of senders arguing that the SMTP response is just for them, at the expense of everyone else who might have got some use out of it? The rest of your message tries to show an equivalence between Michael deliberately temporarily rejecting an email (as opposed to because of some unexpected problem) and SendGrid deciding to discard it without any retries. I've mentioned before that I'm not really interested in debating that one because my main concern here is the unintended consequences of this practice. Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
On Sun, 25 Jun 2023 14:28:33 +0100, Dmytro Homoniuk via mailop wrote: >*In a very non-confrontational way* I want to express my opinion and to >note that's pretty much how senders operate right now: too often the smtp >code and enhanced code the recipient system returns have nothing to do with >what the response text says - and every mailbox provider uses their own >flavor to boot. My favourite is a particular immense mailbox provider that is known to issue a 4xx code with text that basically says "This is a permanent temporary failure; retries will never succeed." Looking for design intent in this phenomenon, one possible objective immediately comes to mind: If you consider the sender to be hostile, issue permanent tempfails so that, if the sender has used a default (long) queue expiration time in their sending software, one can keep a full day or more of re-queued messages on the sender's system, perhaps blowing out the system's disk space, and causing other sending performance issues that reduce the spammer's ability to spam. I have seen these effects in more than one system in the wild. mdr -- "There are no laws here, only agreements." -- Masahiko ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
On Sun, 2023-06-25 at 14:28 +0100, Dmytro Homoniuk via mailop wrote: > > 450 4.3.2 Please retry immediately. If your message was rejected by a > blacklist, see for more information. > > Now this just adds needless ambiguity: was it rejected because of the > blacklist or not? Am I on a blacklist? Which one? Should I actually retry? > And if I retry and see the same thing - does THAT mean I'm blacklisted? > Your message wasn't rejected at all, it was deferred. You're trying to out-think the RFC. > The ESP decided not to retry which was not in user's > best interests, the receiving MTA decided not to accept on first try > - which was also not in user's best interests. I don't see either a > winner here, nor anyone being right really - except the user who is > being royally screwed by everyone Except the part where I decided not to accept the message. I'm actually not sitting in front of the mail server with a button that 4xx defers messages I don't like. Temporary issues happen from time to time. And if you were paying attention at all, you would know that I've been going out of my way for YEARS to make sure that messages like these are received. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
On 6/24/23 14:03, Andy Smith via mailop wrote: If this sort of thing is common amongst large senders, does that mean that we should all be combing our 4xx responses for "triggering" words like "blacklist" and "blocklist"? And how are we to keep up to date with the heuristics of multiple senders? *In a very non-confrontational way* I want to express my opinion and to note that's pretty much how senders operate right now: too often the smtp code and enhanced code the recipient system returns have nothing to do with what the response text says - and every mailbox provider uses their own flavor to boot. It's a bit of a nightmare, really. Not exactly for the reasons this thread was started, but still I find it sometimes too complex when classifying data in the aggregated reports. To illustrate how some responses may be providing extra info without it being very useful: Example 1: *(Alex's hypothetical)* 450 4.3.2 Local problem - couldn't query foobar blacklist I do think this very hypothetical example is a bit of an outlier. It's providing non-actionable information to the sending system: it should read this and ... erm... reach out to you and tell you you may have your blacklist API malfunctioning? As as sender I would be very satisfied with *"450 4.3.2 Local problem - retry later"* - this way you'd tell me the deferral is not exactly my fault, it's you and I'm not expected to figure out the issue on my own. And yes, if it persists - I'd be reaching out to ask about it, so if there is anything I can do - I would want it in the response. Example 2: *(if this is indeed how the message went):* 450 4.3.2 Please retry immediately. If your message was rejected by a blacklist, see for more information. Now this just adds needless ambiguity: was it rejected because of the blacklist or not? Am I on a blacklist? Which one? Should I actually retry? And if I retry and see the same thing - does THAT mean I'm blacklisted? Wishful thinking, sure, but would be great to know if a retry is expected immediately, in a minute, in 20 minutes, in 4 hours or tomorrow - in both examples the code and enhanced code are saying something is temporary but the response doesn't actually provide any insight at all as to HOW temporary a thing is. That's where the sender hubs/postmaster pages would come in help - like Yahoo does with their TSS/IPTS errors, but those are also not very popular and not everyone does them well either. *(kudos to Michael, that page is actually very good)* *On a more confrontational tone *(sorry, still not trying to restart a fight, expressing a possibly controversial opinion)*:* *"The main practical reason for blacklisting is lack of resources."* (quote from Michael's page). Frankly I feel what Luke is doing (and maybe other ESPs?) is basically the same. Both sides here are deciding taking back this planet will cost too much resources so I declare exterminatus onto an entire imperial world .. deciding that something would take too much resources and is not worth it. Sure, one may argue it's not up to ESP to decide that an email is not worth it - but the mailbox provider is not in a better moral position because it's not the end user who makes the decision - MBP made the decision for them. The ESP decided not to retry which was not in user's best interests, the receiving MTA decided not to accept on first try - which was also not in user's best interests. I don't see either a winner here, nor anyone being right really - except the user who is being royally screwed by everyone. -- ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
On Fri, Jun 23, 2023 at 11:13:56PM -0500, Al Iverson via mailop wrote: > What if we just got to the heart of the matter and admitted that > greylisting is useless 2023? That feels like a bit of a strawman, insofar as greylisting is *far* from the only reason why a 4xx could be emitted. - Matt ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
On 6/23/2023 9:13 PM, Al Iverson via mailop wrote: What if we just got to the heart of the matter and admitted that greylisting is useless 2023? This prompts wondering whether it's time for the email anti-abuse community to have a discussion about mechanisms that used to be useful but are no longer worth the effort (or that might actually be counter-productive.) The likely benefits will be simplification on the technical and operations side, and possibly better outcomes. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net mast:@dcrocker@mastodon.social ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
It appears that Bill Cole via mailop said: >On 2023-06-24 at 00:13:56 UTC-0400 (Fri, 23 Jun 2023 23:13:56 -0500) >Al Iverson via mailop >is rumored to have said: > >> What if we just got to the heart of the matter and admitted that >> greylisting is useless 2023? > >+1 > >It was always a cute trick that just happened to mostly work but unlike >harmless cute tricks like banner weirdness, it is intentional shotgun >breakage that requires constant tuning. Constant tuning? The last time I made a significant change to my greylister was in 2013 when I added addres fuzzing. Since then it's been on autopilot and works fine. Despite what some ESPs might assert, retry on 4xx has always been part of SMTP and real MTAs retry. I do agree that if you imagine that it's a FUSSP, or that doing it more aggressively will stop more spam, you'll be sorry. But as a simple hack to catch bots that don't retry, it works pretty well. It probably helps that I don't try very hard to be helpful to senders. My greylist response just says 451 4.4.5 Try again later. R's, John ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
Fully agree! You make a good argument. I didn't mean to say that this issue should be ignored entirely. I would argue that their matchers could be more specific for such a drastic action as "drop the message and don't retry". I just don't agree with the whole sentiment that sendgrid has committed some sort of sin against the email gods by looking at something other than just the code. That's the sentiment this email thread is heavily radiating, but it's just not really helpful in addressing the issue. I think their intentions are reasonable, but their execution could be improved. Stating it like that is also just more likely to make someone engange in conversation, if you only attack someone they get defensive and the conversation is over. I do see what you're saying about unintended consequences. Louis On 24/06/2023 15:03, Andy Smith via mailop wrote: Hello, On Sat, Jun 24, 2023 at 10:18:26AM +, Louis Laureys via mailop wrote: I was with you until it was revealed you mention a blacklist in your response. Sendgrid assumes that the words in the response actually have something to do with the reason it's being temporarily rejected The thing that I keep getting hung up on is the unintended consequences. e.g. I was today years old when I learned that if I made my mail server say: 450 4.3.2 Local problem - couldn't query foobar blacklist upon some local error, that could actually be treated as a permanent failure by the sender and the mail would be silently discarded. I assume that "blacklist" is not the only such poison pill in SendGrid's heuristic. If this sort of thing is common amongst large senders, does that mean that we should all be combing our 4xx responses for "triggering" words like "blacklist" and "blocklist"? And how are we to keep up to date with the heuristics of multiple senders? If the senders' heuristics were perfect then no one would notice, as they would only be discarding mail that never would have got delivered anyway. I realise this is not possible to achieve. But if senders are going to do this sort of thing, I think their goal needs to be to keep it as near to unnoticeable as possible, so I don't think that our response as mailbox providers upon a false positive should be "well of course if you are going to use the word 'blacklist' then what did you expect?" That is saying that we accept that the word 'blacklist' should not appear in any context in a 4xx response, which to me is too bold and is only encouraging more of these obscure and unpublished interactions. Cheers, Andy ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
Why take the time just to victim blame? That wasn't my intention. This has turned into a discussion, and I wanted to add my opinion. I'm not really defending them, I just think you're getting very angry about something that doesn't seem all that odd to me personally. The reason I wanted to add my opinion is that it goes against some of the other opinions in this email thread, and your opinion. It can be good to hear different perspectives, especially when they don't match your own. The only reason they won't do the right thing and simply retry is because it saves them money when sending spam. The arguments about scale are smoke and mirrors. I don't disagree, it is about money as always. I just said that at that scale it can make a difference, while at smaller scales it generally won't. So money wise, it clearly makes sense to them. And my assumption is that that's the reason you mostly see this kind of stuff at larger senders. Louis On 24/06/2023 14:21, Michael Orlitzky via mailop wrote: On Sat, 2023-06-24 at 10:18 +, Louis Laureys via mailop wrote: Michael, I was with you until it was revealed you mention a blacklist in your response. Sendgrid assumes that the words in the response actually have something to do with the reason it's being temporarily rejected, which is a fair assumption to make even if it doesn't follow spec. Then using different retry logic based on that reason does make sense at that scale. I mean effort wise, not "we're big so we can do whatever we want" wise. I'm personally no fan of them, but I don't think this behaviour specifically is odd at all. Why take the time just to victim blame? A decade ago, I added some text to our 5xx rejections to try to be helpful. The RFC explicitly guarantees that doing so will have no negative effects. Fuck me I guess? The response does *not* say that they're blacklisted. The only reason they won't do the right thing and simply retry is because it saves them money when sending spam. The arguments about scale are smoke and mirrors. Your opinions are yours, but you're defending billionaire spammers who knowingly breaking the rules because the rest of us pay for it. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
On 2023-06-24 at 00:13:56 UTC-0400 (Fri, 23 Jun 2023 23:13:56 -0500) Al Iverson via mailop is rumored to have said: What if we just got to the heart of the matter and admitted that greylisting is useless 2023? +1 It was always a cute trick that just happened to mostly work but unlike harmless cute tricks like banner weirdness, it is intentional shotgun breakage that requires constant tuning. -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) Not Currently Available For Hire ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
Hello, On Sat, Jun 24, 2023 at 10:18:26AM +, Louis Laureys via mailop wrote: > I was with you until it was revealed you mention a blacklist in > your response. Sendgrid assumes that the words in the response > actually have something to do with the reason it's being > temporarily rejected The thing that I keep getting hung up on is the unintended consequences. e.g. I was today years old when I learned that if I made my mail server say: 450 4.3.2 Local problem - couldn't query foobar blacklist upon some local error, that could actually be treated as a permanent failure by the sender and the mail would be silently discarded. I assume that "blacklist" is not the only such poison pill in SendGrid's heuristic. If this sort of thing is common amongst large senders, does that mean that we should all be combing our 4xx responses for "triggering" words like "blacklist" and "blocklist"? And how are we to keep up to date with the heuristics of multiple senders? If the senders' heuristics were perfect then no one would notice, as they would only be discarding mail that never would have got delivered anyway. I realise this is not possible to achieve. But if senders are going to do this sort of thing, I think their goal needs to be to keep it as near to unnoticeable as possible, so I don't think that our response as mailbox providers upon a false positive should be "well of course if you are going to use the word 'blacklist' then what did you expect?" That is saying that we accept that the word 'blacklist' should not appear in any context in a 4xx response, which to me is too bold and is only encouraging more of these obscure and unpublished interactions. Cheers, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
On Sat, 2023-06-24 at 10:18 +, Louis Laureys via mailop wrote: > Michael, I was with you until it was revealed you mention a blacklist in your > response. Sendgrid assumes that the words in the response actually have > something to do with the reason it's being temporarily rejected, which is a > fair > assumption to make even if it doesn't follow spec. Then using different retry > logic based on that reason does make sense at that scale. I mean effort wise, > not "we're big so we can do whatever we want" wise. I'm personally no fan of > them, but I don't think this behaviour specifically is odd at all. Why take the time just to victim blame? A decade ago, I added some text to our 5xx rejections to try to be helpful. The RFC explicitly guarantees that doing so will have no negative effects. Fuck me I guess? The response does *not* say that they're blacklisted. The only reason they won't do the right thing and simply retry is because it saves them money when sending spam. The arguments about scale are smoke and mirrors. Your opinions are yours, but you're defending billionaire spammers who knowingly breaking the rules because the rest of us pay for it. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
On Sat, 2023-06-24 at 08:42 +, Andy Smith via mailop wrote: > > In the specific case at hand it seems that the OP is greylisting > SendGrid for being on some sort of blocklist, and the specific > mention of "blacklist" in the 4xx response is hitting a SendGrid > heuristic that says "don't bother to retry these", so messages are > actually being lost. I think it has implications beyond greylisting. > We're not greylisting them for being on a blacklist, they'd just get rejected in that case. The blacklist text is there for the people who get a 5xx reject. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
Dňa 24. júna 2023 8:42:59 UTC používateľ Andy Smith via mailop napísal: >I wouldn't go as far as to characterise my belief that 4xx should >always be retried to be as firm as fanaticism or religion. Like I >say, I can see why in some specific cases large senders might not >want to. The combination of seeing 4xx and "blacklist" being enough >though seems like a pretty blunt tool. I'm surprised at that. Of course, there are cases when remote side responds with not right/appropriate code, by mistake or by purpose (as i often see in responses to some DMARC reports for rejected messages). But make "heuristic" on human readable text (after code) have to be done carefully, as free form text is, ehm, free form text. When done wrongly, the "smart rules" quickly becomes "stupid rules"... regards -- Slavko https://www.slavino.sk/ ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
Michael, I was with you until it was revealed you mention a blacklist in your response. Sendgrid assumes that the words in the response actually have something to do with the reason it's being temporarily rejected, which is a fair assumption to make even if it doesn't follow spec. Then using different retry logic based on that reason does make sense at that scale. I mean effort wise, not "we're big so we can do whatever we want" wise. I'm personally no fan of them, but I don't think this behaviour specifically is odd at all. Louis Op zaterdag 24 juni 2023 om 10:42, schreef Andy Smith via mailop: > Hello, > > On Fri, Jun 23, 2023 at 11:13:56PM -0500, Al Iverson via mailop wrote: > > What if we just got to the heart of the matter and admitted that > > greylisting is useless 2023? > > For me that isn't the heart of the matter. I mean it's an > interesting discussion still, but what surprised me about this > discussion is how easy it is to unwittingly hit one of SendGrid's > heuristics here. > > In the specific case at hand it seems that the OP is greylisting > SendGrid for being on some sort of blocklist, and the specific > mention of "blacklist" in the 4xx response is hitting a SendGrid > heuristic that says "don't bother to retry these", so messages are > actually being lost. I think it has implications beyond greylisting. > > When it comes down to SMTP responses I used to be more on the side > of including verbose text as to reasoning rather than just a terse > code or a generic message. This turn of events is pushing me towards > the "terse code" stance because I don't want to unwittingly have my > 4xx be interpreted in other ways. I'm talking about 4xx that have > nothing to do with greylisting here. > > I wouldn't go as far as to characterise my belief that 4xx should > always be retried to be as firm as fanaticism or religion. Like I > say, I can see why in some specific cases large senders might not > want to. The combination of seeing 4xx and "blacklist" being enough > though seems like a pretty blunt tool. I'm surprised at that. > > I don't have an appreciation of the scale but are the rewards on > SendGrid's side really worth the risk? I think if I was doing it I'd > want to be more specific and maybe even try to also match on > recipient domain and/or MX IP or ASN or something. It strikes me > that if there is an actual problem with messages staying in a queue > for days because of perpetual 4xx it's going to be with large > mailbox providers not every enthusiast with a Linux box and a > handful of users. > > Cheers, > Andy > > -- > https://bitfolk.com/ [https://bitfolk.com/] -- No-nonsense VPS hosting > ___ > mailop mailing list > mailop@mailop.org [mailop@mailop.org] > https://list.mailop.org/listinfo/mailop > [https://list.mailop.org/listinfo/mailop]___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
What if we just got to the heart of the matter and admitted that greylisting is useless 2023? It was meant to annoy botnet spammers etc. who, once upon a time, did not have big iron and big disks to be able to queue and retry. That reality basically no longer exists. I tested greylisting myself this year in my own mail-in-a-box setup and found it was just a pain in the ass that delayed password reset emails while I spent too much time trying to whitelist stuff I cared about receiving in a timely fashion. It's a 1990s style configuration option that makes little sense in 2023 and I don't really see any data proving that it actually does anything that inhibits modern spam. The botnets can retry, too. Cheers, Al Iverson -- Al Iverson / Deliverability blogging at www.spamresource.com Subscribe to the weekly newsletter at wombatmail.com/sr.cgi DNS Tools at xnnd.com / (312) 725-0130 / Chicago (Central Time) ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
I thought I was a wild one with testing in production with SpamAssassin rules set to 0 score. That way I could determine the impact without making the impact at all, or without adding resource overhead to other parties just because I was playing around. I think it's a good point here that we all wish everyone would do the same thing, but you can't just get upset at people reacting to others who aren't doing things the way we wish they were, and not also get upset at the people who they're reacting to. In a perfect world we'd all do the same things all of the time, in reality there's nuance and not making adjustments for it is to be willfully ignorant of it. We all do what we have to do at the end of the day. I'm no fan of SpamGrid, I'm actually genuinely surprised to see that anyone works on the project at all anymore. I thought Twilio was just going to milk it until it's revenue dropped below overhead and then repurpose the IPs / sell off the brand. Since, you know, it seems like so little there has changed from an external perspective since this: Sendgrid Under Siege from Hacked Accounts - Krebs on Security [1] But credit where it's due, someone digging into response codes and reacting to the strange ways that everyone else handles their own mail servers, I respect the work. On 2023-06-23 16:28, Michael Orlitzky via mailop wrote: I don't necessarily disagree with this, but here's another perspective. Blacklists are hard to test in production because they rely on a third party. When testing a new config, it's common (at least in postfix land) to switch your reject code from 5xx to 4xx while you see how things play out. Basically, you're giving yourself time to look at the logs and decide if you're losing mail that you don't want to lose. When you're satisfied, you switch the reject code back to 5xx. There are several config parameters designed for this. Links: -- [1] https://krebsonsecurity.com/2020/08/sendgrid-under-siege-from-hacked-accounts/___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
Default behavior is always to retry 4xx if there isn't a rule in place that overrides it. In this case, it was, and still is (i'm working on it) hitting another rule that was put in place to *not* retry a very specific 4xx that would never result in a successful delivery. The full response in question is this: 450 4.3.2 Please retry immediately. If your message was rejected by a blacklist, see http://michael.orlitzky.com/articles/so_youre_blacklisted.xhtml for more information. The bit of text in the string causing the trouble is the word blacklist. Unfortunately, we see a number of 4xx responses that contain the word blacklist and retries never succeed. For example, "450 4.3.2 your message was rejected because you're on a blacklist" is *not *retried because it's useless and *not* retrying is the appropriate thing to do for our customers. The response we are talking about here matches the response code, enhanced code, and contains the word blacklist. It's a rule collision that we can and will solve for. What really confuses me about this discussion is the almost religious fervor around the idea that under no circumstances should a 4xx not be retried and under no circumstances should a 5xx ever be retried. Anyone who has spent more than an hour reviewing SMTP responses for a large-scale mail system knows this is not true. But here we are. Smart, well intended people saying what is known to be false. It's Pravda-esque. It's the Emperor's New Clothes. I'm happy to be the one to say the emperor is naked and take my licks here. On Fri, Jun 23, 2023 at 12:15 PM Andy Smith via mailop wrote: > Hi, > > On Fri, Jun 23, 2023 at 08:03:40PM +0200, Carsten Schiefner via mailop > wrote: > > how about elaborating a bit further on the whats and whys of your setup? > > Maybe some of us could learn something from that, or maybe SendGrid > would consider that to be giving an advantage to competitors. Really > what I am interested in is the justification for not even retrying > once when receiving the "450 4.3.2 Please retry immediately" > response described by the OP. > > If I understand correctly, the OP had experienced missing mail from > SendGrid previously, had asked why there had not been retries on > a 4xx, and was told that SendGrid uses a complex rule set to decide > whether to actually retry for any given 4xx or 5xx. The OP then > tested that by coming up with "450 4.3.2 Please retry immediately", > which also did not receive any retries at all. > > So this implies that if SendGrid sees a "450 4.3.2" response that it > does not otherwise have a special rule for (or is it any 4xx that > there is no rule for?) then discarding the mail without any retries > at all is what happens by default. > > The idea of not retrying at all on certain specific 4xx responses > doesn't sit that well with me, but I can kinda sorta see why a large > sender might want to do that if they were really sure. But here > seems to be the implication that it's actually quite easy to trigger > that behaviour, unless by some stroke of bad luck "450 4.3.2 Please > retry immediately" happened to match an existing rule. > > It would be useful to know if it is the case that if one uses a 4xx > response that SendGrid hasn't seen before, it's going to result in > no actual retries. > > Thanks, > Andy > > -- > https://bitfolk.com/ -- No-nonsense VPS hosting > ___ > mailop mailing list > mailop@mailop.org > https://list.mailop.org/listinfo/mailop > ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
[NOTE: There's absolutely never any need to CC me individually when posting to this or any other list I'm on. Or not on.] On 2023-06-23 at 12:46:58 UTC-0400 (Fri, 23 Jun 2023 09:46:58 -0700) Luke via mailop is rumored to have said: That's right, Bill. It's so simple. All the ESPs and MTAs out there have tools built in to create and customize response handling rules for absolutely no reason. In 30 years of administering mail systems, including a wide variety of MTAs and a couple of MLMs, I've never found it useful to significantly modify the basic error codes or qualitatively vary reactions to them. Not sure how I'd even do that in Postfix, I'm pretty sure it would require hacking the code in ways that it is not designed for. Yes, it's easy to just never retry any 4xx, no, you can't adjust reaction to a 4xx code based on the text part or retry after a 5xx. We actually just do it for fun. We know that every single 4xx should be retried and every single 5xx should not be retried. But we thought it would be neat to have a table with hundreds of custom rules just to make it sound more complicated than it is. Sounds wild, but people do crazy stuff based on unsupportable hunches unencumbered by real data, particularly when it involves the mixing of computer and human behaviors. It would hardly be novel for an ESP to engage in self-deception about what they need to be the essentially arcane and complex lore of how to send bulk email. I've seen a few robust attempts to comprehensively violate the rules on not interpreting the text part of SMTP replies which, under careful analysis, were on-balance net negatives. The very meager discernible benefits could never justify the concrete cost of building and maintaining a system of dozens or hundreds of custom rules. Maybe you do it differently or better, but I'm a skeptic. So, you're right, it's just because we are big and we don't think the rules apply to us. Super productive take you have there. If I ever thought that I was being 'productive' for Twilio I would bill for it. I can tell you've really given this some thought. More than you can imagine... Nothing in the past 25 years has persuaded me that the "ESP" industry is anything but parasitic and harmful, as it is built on fundamental falsehoods. Your sneering is neither original, surprising, nor persuasive. It is clear that you believe you are exempt from the basic rules of SMTP interoperability and I've seen that before, often. My assertion of *why* you believe that is of course just a guess, but in my experience it has always boiled down to scale and a refusal to accept some fundamental truths of email. If you have evidence that it is actually worthwhile for you, you could just say that. My experience includes evidence that it is not. Maybe that has changed in 2023, but I doubt it. -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) Not Currently Available For Hire ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
On Fri, 23 Jun 2023, Carsten Schiefner via mailop wrote: Hi, Luke (& all) - how about elaborating a bit further on the whats and whys of your setup? Because at first sight it is indeed a bit hard to understand why SendGrid may not be in a position to follow the RFCs and the thereof derived and sort of well-established practices of handling server responses. This might actually take a bit of heat out of this thread. And after all, noboy here is really already at the top of their learning curve as I'd assume - and so additional knowledge is quite likely welcome. Luke did give us some examples of what he is up against in February, on the thread "Gmail blocking of good customer" https://list.mailop.org/private/mailop/2023-February/thread.html The first message in this long thread (50+ messages) is https://list.mailop.org/private/mailop/2023-February/024397.html and Luke made about six contributions. Perhaps the most pertinent one was https://list.mailop.org/private/mailop/2023-February/024461.html where Luke said: 4.1.1 Recipient Address Rejected: this one is tough because with some MXs retries result in delivery, in others, it's a dead end. Dynamic rule handling per receiving MX would be awesome, but it would require machine learning to accomplish at scale. which Michael Orlitzky suggested https://list.mailop.org/private/mailop/2023-February/024463.html might be because of "postfix's unverified_recipient_reject_code". I would think that considering the MX when overriding the response code would be a worthwhile addition to try, whether the machine learning is AI-based or hard coded. -- Andrew C. Aitchison Kendal, UK and...@aitchison.me.uk ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
Hi, On Fri, Jun 23, 2023 at 08:03:40PM +0200, Carsten Schiefner via mailop wrote: > how about elaborating a bit further on the whats and whys of your setup? Maybe some of us could learn something from that, or maybe SendGrid would consider that to be giving an advantage to competitors. Really what I am interested in is the justification for not even retrying once when receiving the "450 4.3.2 Please retry immediately" response described by the OP. If I understand correctly, the OP had experienced missing mail from SendGrid previously, had asked why there had not been retries on a 4xx, and was told that SendGrid uses a complex rule set to decide whether to actually retry for any given 4xx or 5xx. The OP then tested that by coming up with "450 4.3.2 Please retry immediately", which also did not receive any retries at all. So this implies that if SendGrid sees a "450 4.3.2" response that it does not otherwise have a special rule for (or is it any 4xx that there is no rule for?) then discarding the mail without any retries at all is what happens by default. The idea of not retrying at all on certain specific 4xx responses doesn't sit that well with me, but I can kinda sorta see why a large sender might want to do that if they were really sure. But here seems to be the implication that it's actually quite easy to trigger that behaviour, unless by some stroke of bad luck "450 4.3.2 Please retry immediately" happened to match an existing rule. It would be useful to know if it is the case that if one uses a 4xx response that SendGrid hasn't seen before, it's going to result in no actual retries. Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
Hi, Luke (& all) - how about elaborating a bit further on the whats and whys of your setup? Because at first sight it is indeed a bit hard to understand why SendGrid may not be in a position to follow the RFCs and the thereof derived and sort of well-established practices of handling server responses. This might actually take a bit of heat out of this thread. And after all, noboy here is really already at the top of their learning curve as I'd assume - and so additional knowledge is quite likely welcome. Best, -C. -- Von meiner Hängematte aus gesendet. -Original Message- From: Luke via mailop To: Bill Cole Cc: Luke via mailop Sent: Fr., 23 Juni 2023 18:53 Subject: Re: [mailop] SendGrid is deleting your mail That's right, Bill. It's so simple. All the ESPs and MTAs out there have tools built in to create and customize response handling rules for absolutely no reason. We actually just do it for fun. We know that every single 4xx should be retried and every single 5xx should not be retried. But we thought it would be neat to have a table with hundreds of custom rules just to make it sound more complicated than it is. So, you're right, it's just because we are big and we don't think the rules apply to us. Super productive take you have there. I can tell you've really given this some thought. On Fri, Jun 23, 2023, 7:37 AM Bill Cole via mailop wrote: > On 2023-06-22 at 19:14:15 UTC-0400 (Thu, 22 Jun 2023 16:14:15 -0700) > Luke via mailop > is rumored to have said: > > > > Unfortunately we cant make a rule that retries all 4xx and doesn't > > retry > > all 5xx. Despite very smart, well intended people writing RFCs and > > other > > specs that make this feel super clean cut in an acedemic or lab > > environment, it's just not how it works in the wild. > > And yet somehow it is how the overwhelming majority of mail systems have > dealt with SMTP reply codes, for decades, successfully. > > SendGrid's deliverability problems are precisely because you and your > coworkers think that you are so big that the rules can't apply to you. > That you are somehow different and special. > > You are wrong. You're nothing special. You're just big. Get over > yourselves. > > > > -- > Bill Cole > b...@scconsult.com or billc...@apache.org > (AKA @grumpybozo and many *@billmail.scconsult.com addresses) > Not Currently Available For Hire > ___ > mailop mailing list > mailop@mailop.org > https://list.mailop.org/listinfo/mailop > ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
That's right, Bill. It's so simple. All the ESPs and MTAs out there have tools built in to create and customize response handling rules for absolutely no reason. We actually just do it for fun. We know that every single 4xx should be retried and every single 5xx should not be retried. But we thought it would be neat to have a table with hundreds of custom rules just to make it sound more complicated than it is. So, you're right, it's just because we are big and we don't think the rules apply to us. Super productive take you have there. I can tell you've really given this some thought. On Fri, Jun 23, 2023, 7:37 AM Bill Cole via mailop wrote: > On 2023-06-22 at 19:14:15 UTC-0400 (Thu, 22 Jun 2023 16:14:15 -0700) > Luke via mailop > is rumored to have said: > > > > Unfortunately we cant make a rule that retries all 4xx and doesn't > > retry > > all 5xx. Despite very smart, well intended people writing RFCs and > > other > > specs that make this feel super clean cut in an acedemic or lab > > environment, it's just not how it works in the wild. > > And yet somehow it is how the overwhelming majority of mail systems have > dealt with SMTP reply codes, for decades, successfully. > > SendGrid's deliverability problems are precisely because you and your > coworkers think that you are so big that the rules can't apply to you. > That you are somehow different and special. > > You are wrong. You're nothing special. You're just big. Get over > yourselves. > > > > -- > Bill Cole > b...@scconsult.com or billc...@apache.org > (AKA @grumpybozo and many *@billmail.scconsult.com addresses) > Not Currently Available For Hire > ___ > mailop mailing list > mailop@mailop.org > https://list.mailop.org/listinfo/mailop > ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
That's a very silly interpretation of what was said. On Thu, Jun 22, 2023, 11:01 PM Slavko via mailop wrote: > Dňa 22. júna 2023 23:14:15 UTC používateľ Luke via mailop < > mailop@mailop.org> napísal: > > >Unfortunately we cant make a rule that retries all 4xx and doesn't retry > >all 5xx. > > In other words, you are not able to ensure delivery, if target rejects with > 4xx code, and you are not able to stop resend, if target rejects with 5xx > code. > > And this all just because you have big amount of messages and smart > rules? > > Something is rotten in the state of Denmark... > > regards > > > -- > Slavko > https://www.slavino.sk/ > ___ > mailop mailing list > mailop@mailop.org > https://list.mailop.org/listinfo/mailop > ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
On 2023-06-22 at 19:14:15 UTC-0400 (Thu, 22 Jun 2023 16:14:15 -0700) Luke via mailop is rumored to have said: Unfortunately we cant make a rule that retries all 4xx and doesn't retry all 5xx. Despite very smart, well intended people writing RFCs and other specs that make this feel super clean cut in an acedemic or lab environment, it's just not how it works in the wild. And yet somehow it is how the overwhelming majority of mail systems have dealt with SMTP reply codes, for decades, successfully. SendGrid's deliverability problems are precisely because you and your coworkers think that you are so big that the rules can't apply to you. That you are somehow different and special. You are wrong. You're nothing special. You're just big. Get over yourselves. -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) Not Currently Available For Hire ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
On Thu, 2023-06-22 at 16:14 -0700, Luke wrote: > It's specifically targeting the response code and string you provided: 450 > 4.3.2 Please retry immediately Ok, I've spent the morning patching out a few more of our MTA's 4xx responses with that exact text. Some of them contained important diagnostic information, so I'm sabotaging my coworkers (and everyone else who sends us mail) to make this work. Supposing it does, it will have fixed the problem for one person, with one MTA, with one set of custom patches. You're still deleting everyone else's mail. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
Dnia 22.06.2023 o godz. 16:14:15 Luke via mailop pisze: > > Unfortunately we cant make a rule that retries all 4xx and doesn't retry > all 5xx. Why? 4xx means the target says "I want you to retry". 5xx means the target says "I don't want you to retry". Regardless if this is deliberate, or by mistake on target's side, why can't you just be a good netizen and follow it? -- Regards, Jaroslaw Rafa r...@rafa.eu.org -- "In a million years, when kids go to school, they're gonna know: once there was a Hushpuppy, and she lived with her daddy in the Bathtub." ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
Dňa 22. júna 2023 23:14:15 UTC používateľ Luke via mailop napísal: >Unfortunately we cant make a rule that retries all 4xx and doesn't retry >all 5xx. In other words, you are not able to ensure delivery, if target rejects with 4xx code, and you are not able to stop resend, if target rejects with 5xx code. And this all just because you have big amount of messages and smart rules? Something is rotten in the state of Denmark... regards -- Slavko https://www.slavino.sk/ ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
It's specifically targeting the response code and string you provided: 450 4.3.2 Please retry immediately Unfortunately we cant make a rule that retries all 4xx and doesn't retry all 5xx. Despite very smart, well intended people writing RFCs and other specs that make this feel super clean cut in an acedemic or lab environment, it's just not how it works in the wild. So we have rules that sometimes collide with each other. An "obvious" erorr is often lost in a sea of 30 million unique response codes and strings we see every single day. Well-intended rules fall short of perfection. Your ~300 mishandled messages per week is someone else's 300 million messages per week that are properly handled. Don't delude yourself into thinking this is some zero sum game where there are good guys and bad guys. Reality is rarely that simple. But we are here to help if well can. Luke On Thu, Jun 22, 2023, 12:10 PM Michael Orlitzky via mailop < mailop@mailop.org> wrote: > On Thu, 2023-06-22 at 10:52 -0700, Luke via mailop wrote: > > I got a rule in place that should cover this. I'll monitor and make sure > > the desired behavior is occurring. > > > > Thanks, but which "this"? Should all 4xx replies now get a retry, or > just the ones containing a certain phrase? (And which phrase, in that > case?) > > > ___ > mailop mailing list > mailop@mailop.org > https://list.mailop.org/listinfo/mailop > ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
On Thu, 2023-06-22 at 10:52 -0700, Luke via mailop wrote: > I got a rule in place that should cover this. I'll monitor and make sure > the desired behavior is occurring. > Thanks, but which "this"? Should all 4xx replies now get a retry, or just the ones containing a certain phrase? (And which phrase, in that case?) ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
I got a rule in place that should cover this. I'll monitor and make sure the desired behavior is occurring. On Wed, Jun 21, 2023, 7:27 PM Michael Orlitzky wrote: > On 2023-06-21 18:39:59, Luke wrote: > > Ahh. Thats funny. Apologies. You'll have to refresh my memory. > > > > Happy to help if I can. Give me the full response, code and string, and > > tell me how you'd like us to handle it and if it makes sense, I can make > > the necessary changes. > > > > In this case, before patching, the original response was, > > 450 4.3.2 Service currently unavailable > > After patching, it's > > 450 4.3.2 Please retry immediately > > I'd rather not patch our MTA for the rest of eternity, so a rule for > the first response would be preferable. But if that's not possible I > can live with the second. After all, that was my hope/plan after the > last discussion. > > In either case, one quick retry (a minute?) followed by a few slower > attempts (1h, 12h, 24h, etc.) for 4-5 days would be perfect. The > quicker retries help with the planned hiccups, while the longer ones > accomodate the unplanned ones. > > However, a custom rule would address only that one source of 4xx > error. Others are popular. For example, > > 450 4.3.0 Queue file write error > > when we kill a filter process while the MTA is waiting on it. And > that's just for our MTA. > > I think I've already disproven the null hypothesis, that SendGrid > retries normally in the absense of statistical evidence discouraging > it. If that's not the case, it should be: the default should be to do > the right thing and retry. Otherwise, everyone is going to have to hack > their MTAs to return some magic phrase. > > Still, I'll take what I can get. Thanks for responding again. > ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
Am 22.06.23 um 06:52 schrieb Matt Harris via mailop: On Wed, Jun 21, 2023 at 6:11 PM Sebastian Nielsen via mailop wrote: >>The RFC forbids doing that, and I argued against it The RFC and reality is two different things. If a client don't want to retry, I think they are free to choose to not retry. This is a terrible take, imho. It's anathema to building standards-compliant and hence interoperable systems that allow the internet as a whole to function correctly while each of us make different choices in terms of providers, hardware, and software to use. Sometimes deviating from RFCs may be necessary to ensure stable and safe operations, because the RFC authors did not foresee all circumstances and every way malicious parties could try to exploit the system. However, as soon as you deviate from the RFCs you must accept responsibility for the resulting effects on legitimate use. If SendGrid decides to react in a non-standard way to 4xx SMTP responses, they are fully responsible for lost tickets, order and invoice information etc. that might have been sent through their service. For newsletters and promotional mail this is entirely acceptable. For transactional mail with actual value and legal weight, not so much. Cheers, Hans-Martin ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
On Wed, Jun 21, 2023 at 6:11 PM Sebastian Nielsen via mailop < mailop@mailop.org> wrote: > >>The RFC forbids doing that, and I argued against it > > The RFC and reality is two different things. If a client don't want to > retry, I think they are free to choose to not retry. > This is a terrible take, imho. It's anathema to building standards-compliant and hence interoperable systems that allow the internet as a whole to function correctly while each of us make different choices in terms of providers, hardware, and software to use. If someone has a good argument for why non-RFC-compliant behavior is desirable, then they should voice that to the appropriate communities and move towards revising the applicable standards so that everyone and every system can be updated to support that change, not simply go off and do something that creates unexpected and frankly incorrect behaviors and hence breaks the interoperability that we all rely on every day and without which we would have no internet. - mdh Matt Harris VP OF INFRASTRUCTURE Follow us on LinkedIn! matt.har...@netfire.net 816-256-5446 www.netfire.com ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
On 2023-06-22 04:21:59, Sebastian Nielsen via mailop wrote: > >> We update kernels, reload AV signatures, have databases go down, > >> accidentally crash postfix during OS upgrades, typo config files, etc. > > Couldn't you make so if the inner servers are in trouble or go down > or similar, then the perimeter server will buffer the email for you > without a temporary reject? We don't want to accept mail and then later bounce (backscatter) it, so at a minimum, the outer servers need to be able to do recipient validation and spam scanning, and those services can have hiccups. No matter how far you push the problem down the road, there's a fundamental limit you hit eventually. > When it comes to spam and virus filtering, you could skip spam > filtering for "buffered" email, ergo email that was received during > outage. And just apply virus filtering. For virus filtering, you > ARE allowed to silently delete email according to RFC, and what I > know, also for those countries that prohibit silently deleting > emails for "spam": I was referring to a European law that doesn't apply to us, so I don't want to focus on it too much. Instead I'll just say that us silently deleting emails is not a very satisfying solution to the problem of someone else silently deleting emails. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
On 2023-06-21 18:39:59, Luke wrote: > Ahh. Thats funny. Apologies. You'll have to refresh my memory. > > Happy to help if I can. Give me the full response, code and string, and > tell me how you'd like us to handle it and if it makes sense, I can make > the necessary changes. > In this case, before patching, the original response was, 450 4.3.2 Service currently unavailable After patching, it's 450 4.3.2 Please retry immediately I'd rather not patch our MTA for the rest of eternity, so a rule for the first response would be preferable. But if that's not possible I can live with the second. After all, that was my hope/plan after the last discussion. In either case, one quick retry (a minute?) followed by a few slower attempts (1h, 12h, 24h, etc.) for 4-5 days would be perfect. The quicker retries help with the planned hiccups, while the longer ones accomodate the unplanned ones. However, a custom rule would address only that one source of 4xx error. Others are popular. For example, 450 4.3.0 Queue file write error when we kill a filter process while the MTA is waiting on it. And that's just for our MTA. I think I've already disproven the null hypothesis, that SendGrid retries normally in the absense of statistical evidence discouraging it. If that's not the case, it should be: the default should be to do the right thing and retry. Otherwise, everyone is going to have to hack their MTAs to return some magic phrase. Still, I'll take what I can get. Thanks for responding again. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
>> We update kernels, reload AV signatures, have databases go down,
>> accidentally crash postfix during OS upgrades, typo config files, etc.
Couldn't you make so if the inner servers are in trouble or go down or similar,
then the perimeter server will buffer the email for you without a temporary
reject?
Because, what I understand from your description, you have inner postfix
servers, and when these crash due to OS upgrades, they will stop responding on
port 25. Thus the outer perimeter servers return a 4xx code to tell any
would-be mailers that your inner servers has a problem.
By buffering the email in the perimeter servers yourself, you avoid the whole
problem. And for the short time your inner servers have a problem, the outer
servers should have plenty of space to buffer.
When it comes to spam and virus filtering, you could skip spam filtering for
"buffered" email, ergo email that was received during outage. And just apply
virus filtering.
For virus filtering, you ARE allowed to silently delete email according to RFC,
and what I know, also for those countries that prohibit silently deleting
emails for "spam":
"Conversely, if a message is rejected because it is found to contain
hostile content (a decision that is outside the scope of an SMTP
server as defined in this document), rejection ("bounce") messages
SHOULD NOT be sent unless the receiving site is confident that those
messages will be usefully delivered. The preference and default in
these cases is to avoid sending non-delivery messages when the
incoming message is determined to contain hostile content."
Note that with "hostile content" they obviously mean virus attacks, exploits
and such, not phishing or spam that isn't hostile per se, but more a nuisance.
This should create a nice situation for both your customers that don't lose
tickets because SendGrid refuses to retry on 4xx, and for you so you can
upgrade kernels, reload AV signatures, database maintenance, OS upgrades and
change configs without problems.
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
Ahh. Thats funny. Apologies. You'll have to refresh my memory. Happy to help if I can. Give me the full response, code and string, and tell me how you'd like us to handle it and if it makes sense, I can make the necessary changes. On Wed, Jun 21, 2023, 6:31 PM Michael Orlitzky wrote: > On 2023-06-21 18:19:41, Luke via mailop wrote: > > > > I work at sendgrid and manage response handling. If someone were to reach > > out with an obvious problem, I'd be willing and able to adjust our > response > > handling appropriately. > > > > You're the guy I mentioned the problem to last time. > ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
On 2023-06-21 18:19:41, Luke via mailop wrote: > > I work at sendgrid and manage response handling. If someone were to reach > out with an obvious problem, I'd be willing and able to adjust our response > handling appropriately. > You're the guy I mentioned the problem to last time. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
On 2023-06-22 02:05:40, Sebastian Nielsen via mailop wrote: > >> They were going to get a 4xx anyway. I changed the message to *help* > >> SendGrid. > > Yes but if you can change the message for SendGrid only, you can > accept the mail and let it through... Apparently you were able to > send custom text to just SendGrid. Then you have some rule to be > able to differentiate SendGrid mail from other mail. Thus you just > accept it. I was unclear then. I patched the mail server to change the 4xx text that we send to everyone, but I changed it only to help SendGrid. I have no way to know ahead of time who is using SendGrid. Some senders, like GitHub, have dedicated servers and send us enough mail that, over time, I have been able to whitelist all(?) of them. This helps but is not enough: * I can't whitelist a server until we've lost a message from it. The way I find out that there's a server to whitelist is that a customer calls to let us know that he's missing an important piece of mail from it (e.g. the Hershey Park tickets). * Continuing the example, GitHub is constantly adding/changing servers. When they do, we're at risk of losing mail until we can whitelist the new addresses. * The non-dedicated SendGrid servers cannot be whitelisted, because almost all mail from non-dedicated SendGrid servers is spam. * I think you're assuming that most of our 4xx errors are from the spam filter or from overzealous action on my part to make a point; they're not. We update kernels, reload AV signatures, have databases go down, accidentally crash postfix during OS upgrades, typo config files, etc. All of those result in 4xx errors and whitelisting does nothing to help. The bottom line is that avoiding 4xx *entirely* is impossible, even if I could reduce them somewhat at great expense. If nothing else, then in those unavoidable scenarios, you have to agree that SendGrid has a responsibility not to delete important mail. But they do. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
An alternative approach would be to admit that response handling at massive sclae is very difficult to get 100% right. Give the sender the benefit of the doubt that they are trying to do the right thing and attempt to reach someone who works there to see if they can help. You could try mailop, email geeks, maawg slack, linkedin, abuse@, support@. I bet you'd find someone willing to help correct the issue. I work at sendgrid and manage response handling. If someone were to reach out with an obvious problem, I'd be willing and able to adjust our response handling appropriately. But it definitely feels better to just assume the worst and blast people on a community forum. Luke On Wed, Jun 21, 2023, 5:08 PM Sebastian Nielsen via mailop < mailop@mailop.org> wrote: > >> They were going to get a 4xx anyway. I changed the message to *help* > SendGrid. > > Yes but if you can change the message for SendGrid only, you can accept > the mail and let it through > > >> Where do I find out what the IP/domain is? Is it in my mail logs, > > Apparently you were able to send custom text to just SendGrid. > Then you have some rule to be able to differentiate SendGrid mail from > other mail. > Thus you just accept it. > > ___ > mailop mailing list > mailop@mailop.org > https://list.mailop.org/listinfo/mailop > ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
>> They were going to get a 4xx anyway. I changed the message to *help* >> SendGrid. Yes but if you can change the message for SendGrid only, you can accept the mail and let it through >> Where do I find out what the IP/domain is? Is it in my mail logs, Apparently you were able to send custom text to just SendGrid. Then you have some rule to be able to differentiate SendGrid mail from other mail. Thus you just accept it. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
On 2023-06-22 00:32:37, Sebastian Nielsen via mailop wrote: > Why even send retry requests to SendGrid? > Just accept the email, whats the problem? > > If your antivirus or mail scanning solution requires some time, > buffer the email at your server instead. I do understand it creates > the problem of, when deciding to keep or trash the email, SendGrid > client is long gone, And sending back an error would create > backscatter. But then, just silently trash the email if it is > deemed virus or similar. If your spam filter is perfect this can work. Otherwise, silently deleting mail is not nice for our customers, is not nice for the senders, and is illegal in some places. > You could try using stream scanning and then when arriving at final > decision, let client wait for some seconds before getting an scan > result. We already do this, and I don't see how it's relevant. The transaction can still time out, and there are lots of other reasons why we might send a 4xx. > >>These are a pair of tickets for Hershey Park that a family has been waiting > >>on for two days: > > Your fault. You rejected the email with "Please retry immediately" just to > mock with SendGrid. They were going to get a 4xx anyway. I changed the message to *help* SendGrid. > If you have the possibility to set an IP to have custom reject text, > you have the possibility to whitelist that IP (and/or domain) so > mail pass unaffected. Where do I find out what the IP/domain is? Is it in my mail logs, *after* they've deleted the message? Durr ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] SendGrid is deleting your mail
>>The RFC forbids doing that, and I argued against it The RFC and reality is two different things. If a client don't want to retry, I think they are free to choose to not retry. Why even send retry requests to SendGrid? Just accept the email, whats the problem? If your antivirus or mail scanning solution requires some time, buffer the email at your server instead. I do understand it creates the problem of, when deciding to keep or trash the email, SendGrid client is long gone, And sending back an error would create backscatter. But then, just silently trash the email if it is deemed virus or similar. You could try using stream scanning and then when arriving at final decision, let client wait for some seconds before getting an scan result. >>These are a pair of tickets for Hershey Park that a family has been waiting >>on for two days: Your fault. You rejected the email with "Please retry immediately" just to mock with SendGrid. If you have the possibility to set an IP to have custom reject text, you have the possibility to whitelist that IP (and/or domain) so mail pass unaffected. >>SendGrid silently deleted them. We want your mail; If you want their mail, then accept it? Don’t reject with stupid things like "Please retry" for unknown weird reasons? ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
