I would guess, if mail doesn't work it probably relates to spf, dkim or dmark.
Your description matches exactly this: Some service providers gradually increase the rate of rejected mails that come from servers/domains refusing to implement spf and dkim or even fail spf/dkim checks. This can be seen as an incentive so that other service providers harden their infrastructure. At my dayjob we reject all messages that fail spf and dkim checks. Further we actively monitor dmark activity to be informed if third parties are attempting impersonation of company owned domains (aka identity spoofing). Unfortunately you do not get any messages as a "client", be it sender or recipient. Mailserver administrators would see failures, but only if they monitor the right metrics. And, yes, the world is a messy place. ;-) I had to learn some of this craft. Even if I did not master all aspects, I could try my best and do an analysis of some sample message headers? Do not hesitate to send me sample messages that made it into your inbox, please as attachments so that message headers are preserved. On Tue, 15 Oct 2024, 15:25 Ralph Goers, <ralph.go...@dslextreme.com> wrote: > Volkan has taken to pinging me in Slack every time he sends a message to > this list because at least 25% of the time I wasn’t getting his emails. No > one knows why. I checked with my mail provider and have looked everywhere I > have access to. > > Lately I have been getting all of his emails. Again, I don’t know why. > Also, it seemed to only be emails from him. But his emails always made it > to the list. > > I would like email more if it could automatically filter. As it stands I > have something like 60 folders under the email address where I receive > these. That means I have at least 60 filters set up to route my mail into > them. But even that isn’t perfect. GitHub generates so much noise that it > is impossible to keep up with all the notifications it sends me. That is > one of the major reasons I am not in love with the idea of using > discussions. > > Ralph > > > On Oct 15, 2024, at 4:54 AM, Dominik Psenner <dpsen...@gmail.com> wrote: > > > > What concerns me is that email appears to be unreliable where it > shouldn't. > > Either it works always or it doesn't at all. I experience it to be 100% > > reliable. > > > > Short downtimes are OK and to be expected while systems are patched. But > > then email are delivered later. The technology is much like a postman who > > comes back later if he finds the front gate closed. The postman should > not > > burn the letter in front of the gate. > > > > On Mon, 14 Oct 2024, 22:56 Matt Sicker, <m...@musigma.org> wrote: > > > >> That’s because a lot of other things are also using Slack. On the other > >> hand, I had to disable notifications from Slack due to people misusing > it > >> to DM me instead of sending emails to the Secretary properly (unrelated > to > >> Log4j). > >> > >>> On Oct 14, 2024, at 15:53, Ralph Goers <ralph.go...@dslextreme.com> > >> wrote: > >>> > >>> I can’t say I agree with that. It didn’t take me very long to get used > >> to using Slack. > >>> > >>> Ralph > >>> > >>>> On Oct 14, 2024, at 1:47 PM, Matt Sicker <m...@musigma.org> wrote: > >>>> > >>>> There’s a very, very small chance I’ll ever remember to visit a > website > >> to find out about what are essentially emails that could have been sent > to > >> me. I have a regular habit of reading email nearly every day, but > >> developing new habits is unlikely to stick. > >>>> > >>>>> On Oct 14, 2024, at 14:50, Ralph Goers <ralph.go...@dslextreme.com> > >> wrote: > >>>>> > >>>>> Maybe we just need to start contributing to PonyMail to improve the > UI > >> to eliminate actually needing the email delivered to our accounts. > >>>>> > >>>>> I am only 1/4 serious about this. There has to be a better solution. > >>>>> > >>>>> Ralph > >>>>> > >>>>>> On Oct 14, 2024, at 10:25 AM, Matt Sicker <m...@musigma.org> wrote: > >>>>>> > >>>>>> I didn’t get the original email in this thread once again, so I > think > >> I’d support trying somewhere else to host discussions. Besides archiving > >> those messages into a mailing list, it would be great if the solution > >> provided allowed for email interactivity (e.g., you can reply to the > >> notification of a message and it’s added to the thread appropriately; > this > >> is how GitHub notification emails typically work). > >>>>>> > >>>>>>> On Oct 10, 2024, at 05:40, Christian Grobmeier < > grobme...@apache.org> > >> wrote: > >>>>>>> > >>>>>>> Hi > >>>>>>> > >>>>>>> I am generally open to such experiments. I would start with the > >> easiest parts, such as users@, and see where it goes. > >>>>>>> > >>>>>>> I would advise against mirroring it to users@ behind the scenes, > as > >> it may cause privacy problems (we need user consensus to mirror it). > When a > >> user uses GitHub, they know what to expect. > >>>>>>> > >>>>>>> As for Discourse, many use that now, but I find it very > overwhelming > >> and stressful. I prefer the clean Github discussions approach. > >>>>>>> > >>>>>>> I haven't checked against ASF policies but feel positive about this > >> move for the arguments you have given > >>>>>>> > >>>>>>> Kind regards > >>>>>>> Christian > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> On Thu, Oct 10, 2024, at 10:58, Volkan Yazıcı wrote: > >>>>>>>> *Abstract:* Modern email system security measures make it > >> practically > >>>>>>>> impossible for mailing lists to work – many subscribers don't get > >> all > >>>>>>>> emails. This not only hinders communication, but blocks an > inclusive > >>>>>>>> one. *Shall > >>>>>>>> we, as Logging Services, experiment with alternatives?* > >>>>>>>> > >>>>>>>> *Motivation #1: mailing lists technically don't work* > >>>>>>>> > >>>>>>>> Several widely-used email providers (GMail, Yahoo, iCloud, etc.) > >> have in > >>>>>>>> the last couple of years enabled new measures (DMARC, SPF, DKIM, > >> etc.) to > >>>>>>>> verify the authenticity of emails. In short, these measures enrich > >> email > >>>>>>>> content with checksums and signatures capturing its authenticity. > >> When a > >>>>>>>> mailing list system (e.g., ezmlm, mailman) receives such an email, > >> it > >>>>>>>> performs several changes on its content (adds information about > the > >> mailing > >>>>>>>> list, etc.), and delivers it to all subscribers. When the mail > >> server of a > >>>>>>>> subscriber receives such tampered mail, and if that mail server > >> happens to > >>>>>>>> have earlier shared authenticity checks enabled, it discards the > >> email, or > >>>>>>>> at best, marks it as spam. > >>>>>>>> > >>>>>>>> Ralph, Matt, Piotr stated many times that they don't receive all > >> emails. > >>>>>>>> Ralph actually stated many ASF mailing list emails end up in his > >> spam > >>>>>>>> box > >>>>>>>> < > >> > https://the-asf.slack.com/archives/CBX4TSBQ8/p1728032221080189?thread_ts=1727958807.348019&cid=CBX4TSBQ8 > >>> . > >>>>>>>> Recently we witnessed even Brian Proffitt (VP, Marketing & > >> Publicity) > >>>>>>>> suffer > >>>>>>>> from the same problem > >>>>>>>> <https://lists.apache.org/thread/yfmrpjslcbo5jmsqqpvtok1o6lht11rb > >. > >> > >>>>>>>> INFRA > >>>>>>>> is crawling with related tickets: INFRA-24574 > >>>>>>>> <https://issues.apache.org/jira/browse/INFRA-24574>, INFRA-24790 > >>>>>>>> <https://issues.apache.org/jira/browse/INFRA-24790>, INFRA-24845 > >>>>>>>> <https://issues.apache.org/jira/browse/INFRA-24845>, INFRA-24850 > >>>>>>>> <https://issues.apache.org/jira/browse/INFRA-24850>, INFRA-24872 > >>>>>>>> <https://issues.apache.org/jira/browse/INFRA-24872>, INFRA-25947 > >>>>>>>> <https://issues.apache.org/jira/browse/INFRA-25947>, INFRA-26171 > >>>>>>>> <https://issues.apache.org/jira/browse/INFRA-26171> – there are > >> dozens > >>>>>>>> more. > >>>>>>>> > >>>>>>>> This technical difficulty is not only known to us. AFAIK, this is > >> one of > >>>>>>>> the main reasons PSF (Python Software Foundation) decided to > switch > >> from > >>>>>>>> mailing lists to Discourse. Mailman documents several DMARC > >> mitigations > >>>>>>>> < > >> > https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/handlers/docs/dmarc-mitigations.html > >>> , > >>>>>>>> but I think these are workarounds/hacks rather than > well-established > >>>>>>>> solutions. > >>>>>>>> > >>>>>>>> *Motivation #2: ezmlm is dead* > >>>>>>>> > >>>>>>>> ezmlm, the mailing list software ASF uses, is dead – it is neither > >>>>>>>> developed, nor maintained anymore. (Last official release was in > >> 1997, > >>>>>>>> there > >>>>>>>> is the `ezmlm-idx` add-on, which later on became a successor > >>>>>>>> < > >> > https://untroubled.org/ezmlm/faq/What-is-the-difference-between-ezmlm-and-ezmlm_002didx_003f.html > >>> , > >>>>>>>> which last produced a release in 2014, and so on. Long, dead > >> story.) > >>>>>>>> INFRA > >>>>>>>> maintains a very big, sophisticated set of Perl rules for running > >> ASF > >>>>>>>> ezmlm > >>>>>>>> instances. If you look closely at the INFRA tickets I cited above, > >> some > >>>>>>>> suggest INFRA to fork ezmlm and fix some long standing bugs, etc. > >> We can > >>>>>>>> discuss the possibility of migrating from ezmlm to mailman (yet > >> another > >>>>>>>> mailing list software, but one that is still maintained), whether > >> such a > >>>>>>>> migration should be practiced ASF-wide or only for Logging > >> Services, > >>>>>>>> etc. > >>>>>>>> But eventually, we will still be using a mailing list, and as I > >> tried to > >>>>>>>> explain above, IMO, they just don't work good. > >>>>>>>> > >>>>>>>> *Proposal #1: Experimenting with GitHub Discussions* > >>>>>>>> > >>>>>>>> GitHub is our development bread and butter. We use its tickets, > PRs, > >>>>>>>> reviews, discussions, CI, security & code quality checks, etc. It > >> works > >>>>>>>> perfectly and components are integrated well, i.e., you can link > >> issues, > >>>>>>>> comments, PRs, CI runs, etc. Users like it too – we all witnessed > >> the > >>>>>>>> sudden increase in user interactions after migrating to GitHub > >> Issues > >>>>>>>> and > >>>>>>>> Discussions. We can configure sections & categories in Discussions > >>>>>>>> < > >> > https://docs.github.com/en/discussions/managing-discussions-for-your-community/managing-categories-for-discussions > >>> > >>>>>>>> to make it serve as our main communication medium. It also > provides > >> mail > >>>>>>>> notifications and the possibility to respond to them for those who > >> still > >>>>>>>> prefer their email client over a browser. > >>>>>>>> > >>>>>>>> In short, we can quickly configure Discussions, update our support > >> policy > >>>>>>>> page, and start experimenting with it. > >>>>>>>> > >>>>>>>> One can raise the argument that what if Discussions disappear? We > >> can > >>>>>>>> mirror communication there to a mailing list to be on the safe > >> side. Yet, > >>>>>>>> we need to evaluate the necessity of this. > >>>>>>>> > >>>>>>>> *Proposal #2: Experimenting with Discourse* > >>>>>>>> > >>>>>>>> We can get a VM from INFRA and manage our Discourse instance. > >> Though, > >>>>>>>> AFAIC, this will result in a "GitHub Discussions"-like setup with > >> all the > >>>>>>>> integration goodies missing and added server maintenance burden. > >>>>>>>> > >>>>>>>> *F.A.Q.* > >>>>>>>> > >>>>>>>> *What if GitHub Discussions disappear?* > >>>>>>>> > >>>>>>>> In such a case, I presume they will allow us to download the > >> existing > >>>>>>>> archives. In the worst case, we can decide to mirror the > >> communication > >>>>>>>> there to a mailing list. Yet, we need to evaluate the necessity of > >> this. In > >>>>>>>> particular, how big of a problem is this at the experimentation > >> stage? > >>>>>>>> > >>>>>>>> *How will private communication work with GitHub Discussions?* > >>>>>>>> > >>>>>>>> We can create private repositories for internal/private > >> communication. > >>>>>>>> For > >>>>>>>> users/researchers wanting to submit & discuss security issues, > they > >> can > >>>>>>>> get > >>>>>>>> in touch with us (either via email to `security@logging` or some > >> other > >>>>>>>> ASF/INFRA mailing list), we can grant them permissions to > >> collaborate > >>>>>>>> privately on a repository security advisory > >>>>>>>> < > >> > https://docs.github.com/en/code-security/security-advisories/working-with-repository-security-advisories/about-repository-security-advisories > >>> > >>>>>>>> . > >>>>>>>> > >>>>>>>> *Don't the ASF legals require mailing lists?* > >>>>>>>> > >>>>>>>> I am aware that several ASF policies require mailing list > >> communication, > >>>>>>>> e.g., for voting and such. I first want to establish a consensus > >> among us, > >>>>>>>> and then pitch to the board for exemption as a pilot. > >>>>>>>> > >>>>>>>> *Shouldn't this proposal be practiced ASF-wide?* > >>>>>>>> > >>>>>>>> This will be a very (very very very, actually!) daunting route to > >> pursue. > >>>>>>>> I'd rather start small, solve our problem first (if we can), and > >> then think > >>>>>>>> about widening the scope. > >>>>>> > >>>>> > >>>> > >>> > >> > >> > >
