[Mailman-Developers] Re: ARC user options
On Wed 14/Sep/2022 06:35:50 +0200 Stephen J. Turnbull wrote: I still think you're going to get pushback on the requirements language, but the analysis is complete and accurate. That sounds good to me. I've posted the new draft version[*]. As all drafts, it is still open to discussions. Meanwhile let me thank you for your essential support. Best Ale -- [*] https://datatracker.ietf.org/doc/draft-vesely-dmarc-mlm-transform/ ___ Mailman-Developers mailing list -- mailman-developers@python.org To unsubscribe send an email to mailman-developers-le...@python.org https://mail.python.org/mailman3/lists/mailman-developers.python.org/ Mailman FAQ: https://wiki.list.org/x/AgA3 Security Policy: https://wiki.list.org/x/QIA9
[Mailman-Developers] Re: ARC user options
On Tue 13/Sep/2022 10:14:12 +0200 Stephen J. Turnbull wrote: Alessandro Vesely writes: Maintaining synchronization of configurations of two lists will be tedious for the admin, or involve relatively complicated coding if we arrange to automatically mirror configuration changes. Couldn't symlink most stuff? I don't think there's anything to symlink. In Mailman 3 all of this configuration information is in an RDBMS like PostgreSQL, and routing of posts and modification of messages (both bodies and headers). It would also be possible to link DB tables, or to define triggers that replicate insert/ update/ delete on a number of tables/ fields. The question is how much more insight than average list keeping would be needed to do it. Would this approach make sense with Mailman 2? I'm not clear how that would work. Would you expand? 1. lis...@example.com has two subscribers: list-a-mu...@example.com list-a-nomu...@example.com List-A-[no]munge accepts subscriptions according to site and list policy. 2. List-A is configured not to allow other subscribers under any circumstances. List-A-[no]munge accept subscribers under the site and list policy. 3. List-A-[no]munge refuse all posts, and advertise List-A as the destination for posts. List-A accepts posts according to site and list policy. If the site policy is to accept posts from subscribers, it needs to inspect the union of sub-lists subscriber sets. How could that be accomplished? 4. List-A-munge gets From munging for all posts, List-A and List-A-nomunge never get From munging. (In theory List-A-munge could do munging only for p=reject posters, but always doing it probably makes it easier for subscribers to maintain their filters.) How difficult is that to set up? I saw some lists deploying a home-brewed From: munging tool. In that case they can control it directly. https://datatracker.ietf.org/doc/html/draft-ietf-dmarc-arc-usage-09 would be the natural home but it's expired, so it doesn't do any harm to have it in your draft. What I dislike of that document is its considering the availability of a global reputation system as a widespread feature of all mail servers, 90% of the email users on the Internet are served by organizations that can afford comprehensive and reasonably accurate reputation databases and update algorithms. (Whether they do bother with accuracy is another question.) So I think it's reasonable to ask "how does a reputation database affect this feature" several times. IMHO, it becomes overly complicated. Domain-based reputation is already fuzzy, and for giant organizations it becomes unmeaningful —they're just too big to block. Now, start gaming all possible combinations of domains. Replaying a modified message is all too easy, and ARC can be ambiguous about who modified what in a message. Yes, you could feed a neural network with that. Will it be reliable? For the rest of us, there are less sophisticated but still useful shared reputation databases (ie, the RBLs), and local databases such as SpamBayes can be useful. DMARC was thought so that From: bank.example can hardly be faked. Allowing fuzzy overrides is much like getting back to content analysis. I'd mark as trusted only a few domains, based on personal knowledge. while only the known giants actually have one. In that respect, ARC is a centripetal protocol, which is why I've been opposing it until this attempt. Everything is centripetal, because the only way we really know how to scale networks while maintaining discoverability is hierarchically. All reasonably decentralized networks have a (usually very expensive) centralized system at their foundation. I don't see ARC as being particularly biased toward centralization, just because powerful reputation systems are expensive. It is not the cost. To have a global knowledge of the Internet you need to have a user base that is statistically relevant with respect to the global population. That is, you have to be Google, or Microsoft, or Yahoo!, ... If anything, it's the opposite for the mailing list community, because it makes it easier for an independent list host construct and maintain its reputation, and should get it better treatment from those with reputation systems. Yes. However, I think that a list that experiments non-munging will be whitelisted sooner by small, personal sites who trust it than by large orgs computing its reputation. 3. The no-munging method [...] Before allowing subscription to a non-munging list, a MLM MAY test that a recipient effectively receives its messages by sending a test message with a broken signature from a domain having p=reject. This would require the MLM site to maintain a separate site. It can be a dummy subdomain, a few lines in a zone file. I'll change that line to "from a (sub)
[Mailman-Developers] Re: ARC user options
Hi Steve, thanks for your precious insights. Some comments inline and a new version: On Sat 10/Sep/2022 10:41:21 +0200 Stephen J. Turnbull wrote: Alessandro Vesely writes: 3. The ARC method This twin-list proposal doesn't depend specifically on ARC. Right. For each list with From: munging enabled, a participating MLM MUST To be honest, I don't think that Mailman would be willing to conform to this. It is the MLM as a whole which has to conform, if it wishes to participate. Not the mailing list software. I haven't seen the rest of your draft, but this section is really more of an Informative/BCP RFC than Standards Track. Experimental, actually. The old version is here: https://datatracker.ietf.org/doc/draft-vesely-dmarc-mlm-transform/ If this option gets traction (ie, some vocal fans, I don't ask for a huge number), then I think Mailman would be willing, and would prefer, to go all the way to recipient-controlled From-munging as you originally suggested. Should find a list wishing to experiment with it. I'm going to ask again to IETF's Dispatch list when the new method is published in the draft. I push ARC as the authentication method because that was the major objection to using Author: (the "simple" method in the old version.) Maintaining synchronization of configurations of two lists will be tedious for the admin, or involve relatively complicated coding if we arrange to automatically mirror configuration changes. Couldn't symlink most stuff? A per-subscriber option for From munging would be simpler to develop and maintain. Sure. That was the first idea. configure a twin list with From: munging disabled. Both lists have the same posting address, but separate subscriber lists. I don't think having the same same posting address is likely to work. Most MLMs probably won't implement at all, because list creators can do it for themselves by having an umbrella list with two subscribers, the twin lists. The twin lists would be configured to refuse subscriptions and posts from non-members. I'm not clear how that would work. Would you expand? Recipient-controlled From-munging would also prevent unnecessary double messages. It's just cleaner. Yes. https://datatracker.ietf.org/doc/html/draft-ietf-dmarc-arc-usage-09 would be the natural home but it's expired, so it doesn't do any harm to have it in your draft. What I dislike of that document is its considering the availability of a global reputation system as a widespread feature of all mail servers, while only the known giants actually have one. In that respect, ARC is a centripetal protocol, which is why I've been opposing it until this attempt. Have you considered reviving that document? No. DMARC local policy overrides is one of the use cases that [RFC8617] provides for ARC. It requires that a DMARC filter be configured to accept the authentication assessments provided by a verified ARC chain when all of the domains involved are marked as trusted. In that case, the filter overrides DMARC policy and acts as if the current Authentication-Results: were the ARC-Authentication-Results: (AAR:) of the first ARC set (i=1). Normally, a MLM SHOULD apply DMARC policy on message arrival, so the first AAR: is expected to have dmarc=pass. This paragraph also seems unrelated to recipient controls on From munging, and is not ARC-specific. It describes the mechanism by which a receiver accepts dmarc=fail using ARC. (W.r.t. other papers, I add that the domains have to be marked as trusted, which would act as a simplified reputation system.) MLMs which in turn trust third party domains, such that they override DMARC failures in posted messages, MUST This "MUST" isn't going to work. The MLM software can't ensure this, and MLM operators will do as they please. communicate the list of trusted domains to subscribers Currently most sites (MTAs) operate content filters and blacklists, but not whitelists, and the MLMs trust their sites. Right. Anyway, I think individual subscribers aren't going to be making those decisions, and won't want to unless they're the kind of person who wants to run their own MTA, I think. This issue it discussed from several angles in the I-D linked above, and their conclusion is always the same: reputation maintenance is hard, and very site-specific, so no recommendations in the I-D. Should I add that it's out of scope to speculate how users can convince their mailbox provider to trust/ whitelist a given MLM? I think you should provide rationales for these recommendations, and for a MUST they have to be very persuasive, I think. Right. And here's the new text: 3. The no-munging method For each list with From: munging enabled, a participating MLM MUST configure the possibility to have From: munging disabled. Depending on the m
[Mailman-Developers] Re: ARC user options
On Tue 06/Sep/2022 15:28:10 +0200 Stephen J. Turnbull wrote: At worst, one could set up two lists, fed by the same stream, one with munging enabled and the other not, letting users subscribe to the one they prefer. To be honest, while I'm at best 50% willing to implement the user option, I could easily be persuaded by a few experiments with the dual list proposal. This could be implemented almost transparently for the subscriber (except at subscription time) by using an umbrella list. That's right. I'm going to add this as another method to get rid of munged From:'s in my draft. Would you give it a review for me, please? Internet-Draft MLM TransformationsSeptember 2022 3. The ARC method For each list with From: munging enabled, a participating MLM MUST configure a twin list with From: munging disabled. Both lists have the same posting address, but separate subscriber lists. Subscribers who think that their mailbox provider runs a suitably configured DMARC filter can subscribe to the twin list. Users subscribed to both lists receive double messages until they unsubscribe or suspend delivery from one of them. DMARC local policy overrides is one of the use cases that [RFC8617] provides for ARC. It requires that a DMARC filter be configured to accept the authentication assessments provided by a verified ARC chain when all of the domains involved are marked as trusted. In that case, the filter overrides DMARC policy and acts as if the current Authentication-Results: were the ARC-Authentication-Results: (AAR:) of the first ARC set (i=1). Normally, a MLM SHOULD apply DMARC policy on message arrival, so the first AAR: is expected to have dmarc=pass. MLMs which in turn trust third party domains, such that they override DMARC failures in posted messages, MUST communicate the list of trusted domains to subscribers when they announce the creation of the twin list, and on subsequent modifications. How users can query the list of domains trusted by their mailbox providers is beyond the scope of this document. Anyway, all the domains possibly trusted by the list MUST be trusted by the user's MTA as well, and by any subsequent MTA in case of forwarding. TIA Ale ___ Mailman-Developers mailing list -- mailman-developers@python.org To unsubscribe send an email to mailman-developers-le...@python.org https://mail.python.org/mailman3/lists/mailman-developers.python.org/ Mailman FAQ: https://wiki.list.org/x/AgA3 Security Policy: https://wiki.list.org/x/QIA9
[Mailman-Developers] Re: ARC user options
On Tue 06/Sep/2022 06:41:36 +0200 Stephen J. Turnbull wrote: Alessandro Vesely writes: On Sun 04/Sep/2022 13:38:39 +0200 Stephen J. Turnbull wrote: I asked bind-users if anyone verifying ARC saw any difference after trusting isc.org. Besides adding ARC sets, bind-users do From: munging, obviously. Nobody saw any difference.> As far as I know ARC adds nothing if you're also doing From munging. The point of ARC is entirely to get rid of From munging. Once you've munged From, your DKIM will be valid and you have DMARC from alignment. The tale goes that large mailbox providers want ARC as a tool to filter mail streams from lists that don't do a good filtering themselves. ARC, they say, allows to attribute reputation correctly. However, I don't think they can tell who a message is from, after it has been munged. In that respect, From: munging hampers ARC. The point is how does Mailman know whether a recipient's MX trusts this particular list. It doesn't. Recipients can do any damned thing they want. In the case of AT and Verizon, pretty much everything they do is damned. They frequently block all traffic from well-behaved lists because there's a spammer in the same netblock, or just randomly. Heck, that sounds similar to a blog I just read: https://cfenollosa.com/blog/after-self-hosting-my-email-for-twenty-three-years-i-have-thrown-in-the-towel-the-oligopoly-has-won.html And what does it do when it knows. Some people babble something about DNS records, which looks difficult. Another possibility could be an SMTP extension, difficult to implement as it involves multiple levels. I really don't see the bad actors in this space (by which I mean the ISP-based freemail providers) doing either, since they don't even send "we hate you spammer!" DSNs, they just black hole your list traffic and any attempt to complain about it. If services sent honest DSNs, we could discount bogus 550s and not count them as bounces. A MLM would continue From: munging unless a receiver is able to tell it to not do it. An easy way would be to ask the subscriber whether to do From: munging or not. Then, a user can disable From: munging for the messages destined to her. That's easy for those who run their own MTA. People using Gmail, say, would have to figure out, presumably by trial and error. I'm dubious. People are going to get their subscriptions disabled by experimenting. I don't know how many people still want non-personalized lists, but implementing that would require a bit of effort since two messages would need to be prepared depending on whether don't-munge-for-me was set. I see that archived messages are not munged. How come? Isn't the archive a regular subscriber? At worst, one could set up two lists, fed by the same stream, one with munging enabled and the other not, letting users subscribe to the one they prefer. If such an option is not given, a mailing list could add the Author: header field defined by RFC9057. Receivers could restore From: after DMARC filtering. This can't work. DMARC v2 will quickly be forced to check Author alignment. A list can set the Author: header field by copying From:. In the rare case when Author: is already set and differs from From:, it has to be checked. I guess that case is so rare that the list can handle it by sending such messages to the moderator queue. Alternatively, apply DMARC to the Author: domain. That's the "simple" de-munging method in my draft: https://datatracker.ietf.org/doc/html/draft-vesely-dmarc-mlm-transform From the RFC: In that regard, it would be reasonable for an MUA that would normally organize, filter, or display information based on the From: field to give the Author: header field preference. MUAs won't notice Author: fields any time soon. Translated into Scum-of-the-Earth-Spammer: You can use this header to send "referred by someone in victim's contact list" (that you stole from Yahoo) spam and it will bypass DMARC v1 because you can use an aligned From. All-Hail-Author! They're doing that with From:, and it works fine. It is very hard for MUAs to tell spoofed From:'s, and munging doesn't help. Then, some people hold that even writing THIS IS PHISHING loud and clear won't prevent users from opening and clicking that link. Darwin will tell... That assuming that someone is willing to do something to avoid munged From:'s, which I'm beginning to doubt. If ISPs cared at all about their customers, they'd implement ARC and be fine. It completely solves the problem by putting it on mailing lists and other mediators to filter spam before sealing. Implementing and deploying ARC is not enough, you also need to trust the ARC signer/ sealer. ARC won't be effective until it has been deployed at more than 60% of SMTP servers and that's not a problem. :-) https://www.rhyolite.com/anti-spam/you
[Mailman-Developers] Re: ARC user options
Hi Steve! On Sun 04/Sep/2022 13:38:39 +0200 Stephen J. Turnbull wrote: Alessandro Vesely writes: There is a thread about ARC sealing in bind-users[*]. Not sure what you mean by "sealing". Do you mean they're not implementing the rest of the protocol? They add a complete ARC set. Actually, they add three ARC sets to each message, one at reception (with a full ARC-Authentication-Results), followed by intermediate and final sets. They're applying ARC signatures, although they run Mailman 2. It doesn't seem difficult to implement. It's not. But 1. It's a bad idea to do it in Mailman. 2. It was implemented in Mailman 3 three or four years ago as a proof of concept during the development of ARC. 3. There is a milter available for Postfix and Sendmail from the Trusted Domain Project https://github.com/trusteddomainproject/OpenARC as is the basic implementation which I presume is adaptable to Exim, qmail, and other MTAs. This is the preferred approach, as matter of conformance because it should be implemented by the edge MTA(s), and as a practical matter because Mailman *can't* do SPF since it is never an edge MTA. There is also a pure Python implementation on PyPI, I believe (this is the basis for the Mailman 3 plugin, or maybe it was dkimpy). Thank you for that much needed clarification. I heard someone saying that the IETF was waiting to implement Mailman 3 to use ARC in mailing list... BTW, there is also a Perl implementation of ARC included in Mail::DKIM. It requires trusting the users, though. I don't think so, not any more than any other sign-and-send protocol. I think you misunderstood my question here. What it requires is implementation by recipient domains who trust your host, because if they don't it's 2014 all over again for your subscribers if you have any DMARC p=reject posters. Exactly. I asked bind-users if anyone verifying ARC saw any difference after trusting isc.org. Besides adding ARC sets, bind-users do From: munging, obviously. Nobody saw any difference. The point is how does Mailman know whether a recipient's MX trusts this particular list. And what does it do when it knows. Some people babble something about DNS records, which looks difficult. Another possibility could be an SMTP extension, difficult to implement as it involves multiple levels. An easy way would be to ask the subscriber whether to do From: munging or not. I repeat a possible wording for that option, which would be enabled by default: *From munging*: Set this option to /Disabled/ to receive messages with the original From: line intact. Keep in mind that disabling this option will fail DMARC, so keep it enabled unless your MX either doesn't check DMARC or accepts overrides trusting our ARC sets. Then, a user can disable From: munging for the messages destined to her. That's easy for those who run their own MTA. People using Gmail, say, would have to figure out, presumably by trial and error. If such an option is not given, a mailing list could add the Author: header field defined by RFC9057. Receivers could restore From: after DMARC filtering. That assuming that someone is willing to do something to avoid munged From:'s, which I'm beginning to doubt. Best Ale -- ___ Mailman-Developers mailing list -- mailman-developers@python.org To unsubscribe send an email to mailman-developers-le...@python.org https://mail.python.org/mailman3/lists/mailman-developers.python.org/ Mailman FAQ: https://wiki.list.org/x/AgA3 Security Policy: https://wiki.list.org/x/QIA9
[Mailman-Developers] ARC user options
Hi, There is a thread about ARC sealing in bind-users[*]. They're applying ARC signatures, although they run Mailman 2. The last message hypothesizes a user option like so: *From munging*: Set this option to /Disabled/ to receive messages with the original From: line intact. Keep in mind that disabling this option will fail DMARC, so keep it enabled unless your MTA either doesn't check DMARC or accepts ARC overrides. It doesn't seem difficult to implement. It requires trusting the users, though. Would Mailman implement something like it? Why? Why not? Best Ale -- [*] https://lists.isc.org/pipermail/bind-users/2022-September/106612.html ___ Mailman-Developers mailing list -- mailman-developers@python.org To unsubscribe send an email to mailman-developers-le...@python.org https://mail.python.org/mailman3/lists/mailman-developers.python.org/ Mailman FAQ: https://wiki.list.org/x/AgA3 Security Policy: https://wiki.list.org/x/QIA9
[Mailman-Developers] X-Mailman-Original-DKIM-Signature
Hi, I discovered this just today, after a list I'm subscribed to enabled it. I have a filter which tries to validate original signatures by removing footers and subject tags. Removing "X-Mailman-Original-" is going to be the next addition. I guess the purpose of this option is to spare a dkim=fail result in the recipient's header. However, for documentation purposes[*], I'd be comforted by some document, discussion, or even a crispy comment about the intended usefulness of masking existing signatures. Google found nothing. Any pointer? TIA Ale -- [*] I try and document that original signature validation here: https://tools.ietf.org/html/draft-vesely-dmarc-mlm-transform ___ Mailman-Developers mailing list -- mailman-developers@python.org To unsubscribe send an email to mailman-developers-le...@python.org https://mail.python.org/mailman3/lists/mailman-developers.python.org/ Mailman FAQ: https://wiki.list.org/x/AgA3 Security Policy: https://wiki.list.org/x/QIA9
[Mailman-Developers] Footer settings question: how common is dash-dash-space?
Hi, my footer removal software fails on IETF's "last-call" mailing list because it looks for a line of underscores, whereas that list uses "-- ". That sequence is "the separator line between the body and the signature of a message", according to RFC 3676. However, I had never seen it in mailing list footers before. My experience with mailing lists is limited, so I ask you. Is that sequence common enough to try and catch it in a footer removal function? I asked the list owners, and they answered "We didn't choose these settings when the list was set up! It was probably just the default for that version of Mailman." Is that possible? TIA Ale -- ___ Mailman-Developers mailing list -- mailman-developers@python.org To unsubscribe send an email to mailman-developers-le...@python.org https://mail.python.org/mailman3/lists/mailman-developers.python.org/ Mailman FAQ: https://wiki.list.org/x/AgA3 Security Policy: https://wiki.list.org/x/QIA9
[Mailman-Developers] Mass subscriptions as a DoS attack
Trolls can wreak havoc by subscribing to one or more high volume mailing lists on behalf of a target one. For example, someone could subscribe this list to the Linux kernel mailing list. Everybody would see the confirmation message, but by the time someone realizes the need to unsubscribe, the list will have been flooded, thereby realizing the DoS. Are there mechanisms to prevent that? Best Ale -- ___ Mailman-Developers mailing list -- mailman-developers@python.org To unsubscribe send an email to mailman-developers-le...@python.org https://mail.python.org/mailman3/lists/mailman-developers.python.org/ Mailman FAQ: https://wiki.list.org/x/AgA3 Security Policy: https://wiki.list.org/x/QIA9
[Mailman-Developers] Rewriting the Subject:
Hi all, I've installed my reverting DKIM verifier. Today it failed on a specific message which I also received directly. The reason it failed is the Subject: field was rewritten like so: ORIGINAL: Subject: RE: [dmarc-ietf] A policy for direct mail flows only, was ARC questions REWRITTEN: Subject: Re: [dmarc-ietf] A policy for direct mail flows only, was ARC questions The message is tagged X-Mailman-Version: 2.1.29. Is it Mailman doing that change? I found some code which may seem to be doing something like that in CookHeaders.py: rematch = re.match( '(\s*(RE|AW|SV|VS)\s*(\[\d+\])?\s*:\s*)+', subject, re.I) if rematch: subject = subject[rematch.end():] recolon = 'Re:' It seems I have to save the subject if it matches that string, correct? Best Ale -- ___ Mailman-Developers mailing list -- mailman-developers@python.org To unsubscribe send an email to mailman-developers-le...@python.org https://mail.python.org/mailman3/lists/mailman-developers.python.org/ Mailman FAQ: https://wiki.list.org/x/AgA3 Security Policy: https://wiki.list.org/x/QIA9
[Mailman-Developers] Re: Verifying broken DKIM signatures
On Mon 28/Sep/2020 10:23:03 +0200 Alessandro Vesely wrote: >> Does this happen outside of DMARC mitigation? Can you show examples? > > > I checked a few messages and couldn't find a switched To:. Switched Cc: > seems > to happen when one of the recipients is the list itself, which is then moved > to > the last place. (I try to reproduce that behavior with this message.) Although I have set options to receive both my own messages and duplicates, I only received the postmaster copy of that message. The archive[*] doesn't show the header :-( The original had: Cc: Mailman-Developers@python.org, postmas...@tana.it Best Ale -- [*] https://mail.python.org/archives/list/mailman-developers@python.org/message/VRWI7SZ2EE2WR7ZEPJ6CW3LEPIMDN2PD/ ___ Mailman-Developers mailing list -- mailman-developers@python.org To unsubscribe send an email to mailman-developers-le...@python.org https://mail.python.org/mailman3/lists/mailman-developers.python.org/ Mailman FAQ: https://wiki.list.org/x/AgA3 Security Policy: https://wiki.list.org/x/QIA9
[Mailman-Developers] Re: Verifying broken DKIM signatures
Hi Steve, your observations put me on the right track. Thank you so much! Long post below: On Thu 24/Sep/2020 12:31:29 +0200 Stephen J. Turnbull wrote: Alessandro Vesely writes: First, what Mailman are you talking about? Only Mailman 3 is likely to get these improvements, as Mailman 2 is end-of-life. However, Mailman 2 installations are likely to be around in large numbers for several years, and if Mailman 2 is any evidence, likely few Mailman 3 installations would use these features unless forced to by a disaster like the Yahoo/AOL sudden switch to DMARC p=reject. Reversing transformation should work with any Mailman version, and with other mailing list managers as well. Hence, one cannot rely on some precise indications, but rather on common MLM behavior. Yet, it is possible to undo the transformation that Mailman put in place, thereby validating the original DKIM signature. > It would always be possible to undo all transformations by supplying the original email as part of a multipart/alternative, or perhaps a new multipart subtype, maybe with some kind of device to make reading the message/rfc822 original difficult in standard MUAs. Comparing to the original can make it more difficult to check that the transformed version does not deviate from the original in unacceptable ways. While limiting the cases where original signatures can be recovered, a set of accepted transformations also limits the attack surface. (In the case of Microsoft MUAs, if Mailman is configured to strip HTML, the result might be less than 10% bigger than the original! ;-) Even if plain text is safer, stripping HTML is irreversible. It is sometimes possible to reverse transformations with only the information in the post after Mailman processing. However, some very desirable changes are destructive (eg, anonymized lists, conversion of HTML to plain text, removal of prohibitive attachments). Some non-destructive changes (headers and footers) are highly customizable. So the question is what are the transformations that users want to reverse, and whether that's really possible. It has to be the responsibility of the list owner to configure Mailman so that the transformations can be reversed. Some options, like anonymized lists, are clearly at odds with the need to recover the original author domain's signature. This kind of transformation reversal probably requires no changes to Mailman, just an addition of a Handler which could be written independently and "dropped in" (with a configuration change to the default pipeline). On the other hand, a new header field can be abused. List-*, for example, are often found in a number of messages which don't come from mailing lists, but just aim at not being classified as UCE. The necessary information about transformations that are configured would be available from Mailman in the usual way (existing Handlers need that information). There could be more info than just the confguration, see the heuristic below. Mailman carries out some irreversible changes, such as rewriting To: or Cc: changing the order of the mailboxes, Does this happen outside of DMARC mitigation? Can you show examples? I checked a few messages and couldn't find a switched To:. Switched Cc: seems to happen when one of the recipients is the list itself, which is then moved to the last place. (I try to reproduce that behavior with this message.) or rewriting Content-Transfer-Encoding: irrespective of quotation marks and case (for example "7bit" even if the original, signed field was spelled as "7Bit"). > I'm not aware of such behavior *unless other modifications were done*. In that case, Mailman is specifying the C-T-E it uses, it is not rewriting the original C-T-E. I don't know if it's Mailman or a DKIM signing tool running afterwards, but many plain text messages from mailing lists come rendered as base64. Since the footer is part of the only MIME entity, the reversal has to decode base 64, remove the footer, and re-encode as base64 if that was the original C-T-E. In the latter case, one needs to know the column width of the original encoding... I guess this behavior is coded deeply in Python libraries, I don't think so. As far as I know, the email module in Python 3 provides some support for parsing header fields but I don't know why this would change order or spelling of field contents. Aha, you're right. It is probably Mailman writing its own C-T-E, which happens to be the same as the original, albeit spelt differently. I would guess that to the extent that it happens it has to do with Mailman-level processing (for example, collecting addresses from the same domain so they can be presented as multiple RCPT TO with a single DATA). No, the MTA doesn't care about header addresses. Domain optimization has to be done after MX lookup. I can say for sure that some care was taken to ens
[Mailman-Developers] Verifying broken DKIM signatures
Hi all, there's been quite some discussion about signature breaking in indirect mail flows. Rewriting the From: header field seems to be the most popular workaround. Yet, it is possible to undo the transformation that Mailman put in place, thereby validating the original DKIM signature. I have a few questions about this. That technique is described by the DKIM Transformations draft[*]. It considers a number of reversible transformations, such as adding a footer or adding a mime part containing a footer. I implemented a utility to carry out that task, and it works in a good deal of cases. However, it fails in some cases. Mailman carries out some irreversible changes, such as rewriting To: or Cc: changing the order of the mailboxes, or rewriting Content-Transfer-Encoding: irrespective of quotation marks and case (for example "7bit" even if the original, signed field was spelled as "7Bit"). I guess this behavior is coded deeply in Python libraries, but would like to know developers' opinions. Is that something that could be fixed? The second question is about producing a hint to the verifier telling which transformation(s) have been applied to the message. That would come as an additional header field, for example: DKIM-Transform: footer or as an extra tag in a DKIM signature, for example: DKIM-Signature: v=1; (...) tf=footer; (...) That hint could spare the verifier one pass over the message. Is it something that could be implemented? If not, I'd try guessing, according to this scheme: outermost Content-Type: | first entity Content-Type: | transformation | +-+-+ text/plain | any | footer | +-+-+ multipart/mixed | multipart/mixed | add-part | +-+-+ multipart/mixed + any other | mime-wrap | +-+-+ any other | any | non-reversible | +-+-+ Does that look correct? I know that Mailman has many more features that can hardly be amenable to a fixed set of easily reversible transformations. The point is to be able to preserve DMARC for /some/ mailing lists which care to do so. Currently, there are mailing lists which don't do any change, not even subject tags, in order to avoid breaking DKIM signatures. A somewhat Procrustean solution. I don't think From: rewriting is going to be disabled any time soon. Of course, the original From: must be known in order to validate the original signature. In this respect, I'm hesitant about using Reply-To:. I read in the wiki the original content of From: /may/ be added to Reply-To: or to Cc:. In addition, Reply-To: usually comes after From:, thereby requiring to go back to change already parsed fields. As an alternative, I'd provide for yet another field to be put near the top of the header. Original-From:, say. This may seem redundant, however it serves a different goal. In addition, if the Original-From: is put in place by the original signer, it ratifies its knowledge that From: will be rewritten and its willingness to recover it afterwards. Is this endeavor completely useless, given that the current settings work well enough? Or could it help keeping a consistent DMARC semantics among participants yearning to do so? I'd be glad to hear your opinions... Best Ale -- [*] https://tools.ietf.org/html/draft-kucherawy-dkim-transform ___ Mailman-Developers mailing list -- mailman-developers@python.org To unsubscribe send an email to mailman-developers-le...@python.org https://mail.python.org/mailman3/lists/mailman-developers.python.org/ Mailman FAQ: https://wiki.list.org/x/AgA3 Security Policy: https://wiki.list.org/x/QIA9
[Mailman-Developers] Re: Anonymous mailing lists
On Tue 15/Sep/2020 13:58:24 +0200 Stephen J. Turnbull wrote: ves...@tana.it writes: Someone started talking about the risk of having their names and email addresses archived in a publicly accessible mailing list. So I thought I'd ask. In short, the proposal provided for completely removing such data, to protect privacy. Mark provides the practical stuff. Here's some theory. "Completely" is a really hard problem given that some people obfuscate their addresses in .signatures and in text, and the difficulty of recognizing personal names. There's also the fact that an anonymous list is a flag that you've got people who want to be anonymous. A big bullseye for troublemakers (and the FBI, the MSS, and the GRU). So, in practice, you say that anonymity as a list option is not provided because users don't need/ want it. Correct? As an additional bonus, that would also "solve" any DMARC problem. Sure, but the cost is extremely high. I like to know who's posting. me too. There are other possible DMARC workarounds. I'll be back soon... Best Ale -- ___ Mailman-Developers mailing list -- mailman-developers@python.org To unsubscribe send an email to mailman-developers-le...@python.org https://mail.python.org/mailman3/lists/mailman-developers.python.org/ Mailman FAQ: https://wiki.list.org/x/AgA3 Security Policy: https://wiki.list.org/x/QIA9
Re: [Mailman-Developers] Use of the public suffix list
On Thu 02/Nov/2017 03:31:46 +0100 Stephen J. Turnbull wrote: > Alessandro Vesely writes: > >> * The specs say that "DMARC should be amended to use [a method >> better than PSL] as soon as it is generally available" [1]. I >> believe that sentence refers to RDAP, which was released more or >> less at the same time (March 2015) [2]. >> >> [1] https://tools.ietf.org/html/rfc7489#appendix-A.6 >> [2] https://datatracker.ietf.org/wg/weirds/documents/ > > I see nothing in a quick look at the RDAP spec to suggest that an > organizational/administrative domain (AD) field has been defined. It > seems like it's just intended to be a replacement for whois, of course > allowing extensions like delegating the AD to subdomains (or however > that would work -- it's not obvious to me). That presumably would > either be registered in the RDAP extensions registry or as a separate > RFC. I've seen no discussion of this on DMARC channels either. Yes, RDAP is WHOIS with a machine-readable format. In both cases, a name retrieved from a public domain name registry (DNR) is an "organizational" domain. Rfc7843 defines a publicIds.type to recognize DNRs. That way, one could work out the PSL based on authoritative sources. To make the method practical, RDAP addresses the bootstrap problem (rfc7484), whereby a client could learn DNR info in one shot. To get a feeling of the state of the project, compare the contents of the following URLs --neither of which is usually up to date: https://www.iana.org/domains/root/db http://data.iana.org/rdap/dns.json The second one is mentioned here: https://github.com/arineng/nicinfo/wiki/RDAP-FAQ >> Surprisingly, the publisuffix package itself is not upgraded as >> frequently as the PSL. > > I'm not surprised. Most users of the package won't be upgrading that > frequently either, I suppose, but will rather be downloading it from > the source. PSL take the bother of writing "have your app download the list no more than once per day", so a cron job for each client would do. It is more difficult to coordinate apps, so as to download a single copy for all of them, in the unlikely case that more than one app run on the same host. That way, it becomes a potential packaging problem. > In any case, this isn't a problem for Mailman to deal with; it's easy > enough to access the public suffix list. A site could do that as a > cron job once a day and almost all Mailman subscribers would be > protected due to our "count bounces once per day" algorithm -- only > sites with an extremely low bounce threshold would have a problem. I > suppose there is a backscatter issue, but it's not clear to me that > that is such a big deal. > > This isn't a big deal for us at the moment, and my assessment is that > it will not be one for the forseeable future. With the exception of > WePublished1.3BillionAddressBooksToSpammers!.com and WeDidToo.com, I :-) > haven't heard of anybody publishing p=reject except for domains that > produce only transactional mailflows. I'm sure there are many others, > but I expect that most people will be subscribing to lists with > mailboxes whose domains either have their own _dmarc TXT record or > have an "obvious" administrative domain, or are "p=none" per default. > > Do you have a reason to believe otherwise? No, not really. If anything, I see less and less authenticated mail, both ham and spam. (Possibly an effect of that p=reject?) I know that it is a good thing that DMARC and PSL bring the possibility to learn domain names, but I don't know why. Thank you for sharing your insights Ale -- ___ Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
[Mailman-Developers] Use of the public suffix list
Hi all, I noticed (from a DMARC mitigation utility that Lindsay extracted) that Mailman features its own approach to using the PSL. Of course, development must go on, and sometimes it is a waste of time trying to make a super-duper scaffolding for a job that can be carried out complying to the KISS principle. At any rate, what is the future of DMARC lookups in Mailman? * The specs say that "DMARC should be amended to use [a method better than PSL] as soon as it is generally available" [1]. I believe that sentence refers to RDAP, which was released more or less at the same time (March 2015) [2]. [1] https://tools.ietf.org/html/rfc7489#appendix-A.6 [2] https://datatracker.ietf.org/wg/weirds/documents/ * There are various Python packages for domain name splitting. They obviously use the PSL, but supposedly would transparently switch to a better method in case. If Mailman used one such package, a practical advantage for users would be to update the PSL in only one place, if they happened to use the same dependency. I found six packages. tldextract [3] is the only one of them which caches a JSON object rather than the original textual representation of the list. It uses a frozenset. tld [4] and publicsuffixlist [5] also build a set. publicsuffix[6] and publicsuffix2 [7] build lists of nested dictionaries from all the labels. dnspy [8] builds a dictionary of FQDNs, somewhat like Mailman. How does the time to build the structure compare with the time taken by DSN queries? [3] https://pypi.python.org/pypi/tldextract [4] https://pypi.python.org/pypi/tld [5] https://pypi.python.org/pypi/publicsuffixlist [6] https://pypi.python.org/pypi/publicsuffix [7] https://pypi.python.org/pypi/publicsuffix2 [8] https://pypi.python.org/pypi/dnspy * Debian distributes a publicsuffix package which brings a textual version of the list. Since stretch, it also brings a "dafsa" version. Nowadays, most C implementations (Firefox, Chromium) use dafsa. They build the structure using offsets rather than pointers, so that the blob can be defined in a source file as a literal static array of chars, in order to minimize loading time. That strategy works well as long as the relevant package is upgraded more frequently than the PSL. Otherwise, as for libpsl, one ends up using obsolete data. Surprisingly, the publisuffix package itself is not upgraded as frequently as the PSL. This bug [9] is what prompted me to write this message. I guess you, as Mailman developers, have pondered this subject and I'd be interested to know what you think. [9] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=879008 TIA for any reply Ale -- ___ Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] Camera-ready option to mitigate DMARC issues
On Sun 06/Nov/2016 09:17:53 +0100 Stephen J. Turnbull wrote: Alessandro Vesely writes: The idea is to add a footer only in case it is not present, Aside from the technical difficulties that Mark describes, this suffers from a really big defect: for this to be actually useful, you'd need near-100% participation (Authenticated Received Chain has the same problem). No, the difference is that ARC applies at the receiving side, so you need 100% compliance of list subscribers. Camera-ready applies at the sending side, so the list can still apply any anti-DMARC workarounds if it fails. Once you've got that, no new Mailman code is needed: just don't use a footer in the first place! And it requires effort on the part of providers we already know are irresponsible and inconsiderate because they provide personal mail services disrupted by "p=reject", or on the part of users at those sites, many of whom blame mailing lists, not their irresponsible and/or inexpert providers, for difficulties with posting. Yes, but then it would be less disgusting to punish users of intractable providers. If you don't get near 100%, then you need to provide the workarounds for all users and originating sites that don't participate anyway, to all of your subscribers -- and "nonparticipants" will include the posters who are most likely to blame the lists for any delivery problems, and get upset about "different treatment" (eg, From- munging). Yes, that's already in place. It is considered a somewhat dirty solution. Camera-ready is cleaner than anything I have heard till now. Probably it is not workable, but I cannot understand why. It works well in several publishing environments, typically journals, which distribute templates to authors. Why can't it work for mailing lists? Ale ___ Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] Camera-ready option to mitigate DMARC issues
On Sat 05/Nov/2016 19:51:13 +0100 Mark Sapiro wrote: On 11/05/2016 04:11 AM, Alessandro Vesely wrote: The idea is to add a footer only in case it is not present, similar to what is done with subject_prefix. By properly setting both of them, a sender can submit what can be called a camera-ready post. Since no change applies, no DKIM signature breaks. Hence, dmarc_moderation_action is not needed for such posts. It is not even necessary to check author's domain policy. Mailman could conceivably keep track of whether it has changed any headers or anything in the body of the post, but it's more complicated than that. The first big problem is the Munge From or Wrap Message transformations are applied before any msg_header or msg_footer is added (or maybe added). Is it possible to abort processing the in-memory msg and revert to the file? Doing so --after thorough checks-- would prevent breaking the signature by altering the order of recipient, switching MIME values from token to quoted-string or vice-versa, and similar changes that memory representation may unwittingly imply. I.e. in both MM 2.1 and MM 3, the DMARC mitigations are applied in the incoming handler pipeline before the message is queued for delivery processing. All right, so we cannot save that lookup. Various decorations such as adding msg_header and msg_footer and modifying To: depend on "personalization" and have to be done by delivery processing on a per-recipient basis. In fact, the "camera ready" notion can't apply to any post that is going to be personalized in any way. This in itself would limit the usefulness. Sure, personalization cannot be compatible with camera-ready. It would be more feasible to do this by the poster adding a "X-Camera-Ready:" header to the post saying don't transform my message, but this is unacceptable as it would bypass content filtering, personalization and various other things. X-Camera-Ready: may be useful to automate at senders'. For an author doing it by hands, having to set a header field is an added difficulty... I'm not familiar with Mailman administration, so I ask your opinion. How long would it take to code this option? How many angels can dance on the head of a pin? Ah, not so many of them are still able to perform that, nowadays ;-) Camera-ready posts would be created by hands, by cleverly configuring some email client, or by using purposely written add-ons. They could also be done by MSAs who care about the damage they cause by publishing p=reject --the process can certainly be standardized and automated. How does the sender's automated process even know what msg_header and msg_footer will be added by the list? MTAs can learn List-Post addresses when they receive mail. When they see one, they can, say, change the envelope recipient to an internal mailbox which processes the decoration (and maybe adds X-Camera-Ready:). The decorating module could be carved out from Mailman, and complemented so as to let it download its parameters from well known locations or some such. Ale ___ Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
[Mailman-Developers] Camera-ready option to mitigate DMARC issues
Dear all, I'd like to probe the feasibility of this option. The idea is to add a footer only in case it is not present, similar to what is done with subject_prefix. By properly setting both of them, a sender can submit what can be called a camera-ready post. Since no change applies, no DKIM signature breaks. Hence, dmarc_moderation_action is not needed for such posts. It is not even necessary to check author's domain policy. I'm not familiar with Mailman administration, so I ask your opinion. How long would it take to code this option? How useful would it be? Camera-ready posts would be created by hands, by cleverly configuring some email client, or by using purposely written add-ons. They could also be done by MSAs who care about the damage they cause by publishing p=reject --the process can certainly be standardized and automated. TIA for any reply, and thanks for a great product Ale ___ Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
