Re: [Mailman-Users] Mailman reusing message-id, leads to duplicate message suppression
Barry Warsaw writes: Indeed. It would be great if you could suppress the direct CC, It's not that hard if done in the MUA. My problem is that Uday keeps fiddling with the relevant functions in VM and I get merge conflicts which aren't always trivial to solve. ;-) -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman reusing message-id, leads to duplicate message suppression
* Stephen J. Turnbull step...@xemacs.org: Ralf Hildebrandt writes: Strictly speaking, all that womand wanted was to know if the message passed both mailing lists... So she should have more faith :) For lists hosted by the same Mailman, Mailman 3 might be able to In my case its two lists on the same mailman instance. -- Ralf Hildebrandt Charite Universitätsmedizin Berlin ralf.hildebra...@charite.deCampus Benjamin Franklin http://www.charite.de Hindenburgdamm 30, 12203 Berlin Geschäftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155 -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman reusing message-id, leads to duplicate message suppression
* Mark Sapiro m...@msapiro.net: On 11/05/2013 07:14 AM, Ralf Hildebrandt wrote: Strictly speaking, all that womand wanted was to know if the message passed both mailing lists... So she should have more faith :) In cases like this, I suggest that the user check the list's archive. Good point! Ralf Hildebrandt Charite Universitätsmedizin Berlin ralf.hildebra...@charite.deCampus Benjamin Franklin http://www.charite.de Hindenburgdamm 30, 12203 Berlin Geschäftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155 -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman reusing message-id, leads to duplicate message suppression
* Stephen J. Turnbull step...@xemacs.org: Note that the subject is incorrect. Mailman is not reusing the Message-ID, it is refusing to alter it which is correct behavior according to RFC 5322 (Message-ID is an originator field). I believe that according to RFC 5322 (and predecessors) Mailman SHOULD add a Resent-Message-ID to indicate that it handled the message, but I doubt this would change the duplicate-suppression behavior of Gmail and MS Exchange. I agree. The duplicate-suppression in Exchange uses both Message-ID and Date headers. Footnotes: [1] It's surprisingly difficult to handle duplicates truly gracefully. Ohhh yes. For example, I personally consider the mailing list copy to be the canonical copy of a post, but direct CCs often arrive more quickly. Same here. Happens all the time. -- Ralf Hildebrandt Charite Universitätsmedizin Berlin ralf.hildebra...@charite.deCampus Benjamin Franklin http://www.charite.de Hindenburgdamm 30, 12203 Berlin Geschäftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155 -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman reusing message-id, leads to duplicate message suppression
On Nov 06, 2013, at 09:05 AM, Stephen J. Turnbull wrote: For lists hosted by the same Mailman, Mailman 3 might be able to handle this by adding *both* List-Ids to the header, and only adding the (other) RFC 2369 headers for the list(s) the user is subscribed to. Of course this requires personalization, but that seems to be less and less a problem these days. MM2.1 adds X-BeenThere headers for every list the message flows through; these are to prevent loops. However MM3 got rid of this header in favor of the RFC 2919 List-ID headers. I took a quick look at that RFC just now and I didn't see anything about how many List-Id headers a message could have, but from the discussion it seems to assume there'd only be one header per message. If that solution is interesting to anybody, please let me know, and I'll check the relevant RFCs to see if multiple values are permissible for List-* headers (and for which ones). Code of course will depend on Barry's permission, but I suppose he wouldn't object to providing it as an option. Best to file a bug so we can explore the issue and solutions. You can then attach a branch to the bug. Cheers, -Barry signature.asc Description: PGP signature -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman reusing message-id, leads to duplicate message suppression
On Nov 06, 2013, at 09:51 AM, Ralf Hildebrandt wrote: I believe that according to RFC 5322 (and predecessors) Mailman SHOULD add a Resent-Message-ID to indicate that it handled the message, but I doubt this would change the duplicate-suppression behavior of Gmail and MS Exchange. This probably makes sense, so please submit a bug. Footnotes: [1] It's surprisingly difficult to handle duplicates truly gracefully. Ohhh yes. I've found 'd' in my MUA works about as well as anything. :) For example, I personally consider the mailing list copy to be the canonical copy of a post, but direct CCs often arrive more quickly. Same here. Happens all the time. Indeed. It would be great if you could suppress the direct CC, but of course, that's impossible since Mailman never sees them. The best you can do is suppress the list copy, but that isn't a great solution either if your MUA knows how to do sane things based on the List-* headers. -Barry -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman reusing message-id, leads to duplicate message suppression
In cases like this, I suggest that the user check the list's archive. Only if the list has an archive. --Barry Finkel -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman reusing message-id, leads to duplicate message suppression
On 11/5/13, 4:25 AM, Ralf Hildebrandt wrote: Hi! Mailman re-uses the message-id of the original email. In our case an exchange user is recipient of two mailing-lists. Both mails are adressed in the original message, both lists distribute the list, two emails having the same message-id arrive at the Exchange server, which seems to discard the second mail. Who's to blame? Mailman for re-using the message-id? Exchange for dropping the sencond mail on the floor? This is really a tough problem. Presumably, since mailman doesn't significantly alter the message (which is one reason it is allowed to maintain the message-id) it should really matter that the recipient only gets one copy of the message, or which one. The source of the problem is that maintaining the message-id is important for message references which are used for threading to work well. People who are CC to the message, and use the copy they got directly, and reply back to the list, will want there reply marked as a reply to the list message. This REQUIRES that the message given to the list have the same message-id as the incoming message. The idea of dropping second copies of messages also can make some sense, if you already have a message, do you really need a second copy? This is almost absolutely right if the two copies are due to being listed twice in the distribution, or the duplication due to automatic verbatim forwarding from one address to another. It isn't quite so for sure in the case of mailing list, or if the user wanted to compare things like received headers to compare path ways. I suspect that the people implementing this feature really didn't think about the mailing list case. Note that another use of message-ids, in Usenet, has slightly different rules, and there it is vital that duplicate message-ids be dropped due to the fact that a multi-peered server should normally get many duplicate message-ids from the various peers. But in Usenet, you don't have the case of a central mailing list hub, gathering messages, being allowed to make administrative changes, and then redistributing. I don't think Microsoft is reusing Usenet code, but someone may have been confused by Usenet RFCs, and applied them to email, whose system has a lot of similarity (and even share some RFCs), but IS a different system. In conclusion, the fault probably is on the mail system, but they can probably claim they had good reasons to do it. Hopefully there is a way to turn off the option if really needed. This is also a problem with Google, which does the same thing, with no option to turn it off. The problem more often shows up there by people wonder why their messages don't make it to the list, because Google removed the list distributed one, because the poster had a copy of the message in their outbox. -- Richard Damon -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman reusing message-id, leads to duplicate message suppression
* Richard Damon rich...@damon-family.org: This is really a tough problem. Presumably, since mailman doesn't significantly alter the message (which is one reason it is allowed to maintain the message-id) it should really matter that the recipient only gets one copy of the message, or which one. Yes, only the Subject is slightly being altered. The idea of dropping second copies of messages also can make some sense, if you already have a message, do you really need a second copy? This is almost absolutely right if the two copies are due to being listed twice in the distribution, or the duplication due to automatic verbatim forwarding from one address to another. It isn't quite so for sure in the case of mailing list, or if the user wanted to compare things like received headers to compare path ways. I suspect that the people implementing this feature really didn't think about the mailing list case. Or the users wants to check if the message has indeed been distributed via two or more mailing lists! In conclusion, the fault probably is on the mail system, but they can probably claim they had good reasons to do it. Hopefully there is a way to turn off the option if really needed. There's not way of turning this off on Exchange -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman reusing message-id, leads to duplicate message suppression
--On November 5, 2013 10:25:20 AM +0100 Ralf Hildebrandt ralf.hildebra...@charite.de wrote: Who's to blame? Mailman for re-using the message-id? Exchange for dropping the second mail on the floor? The messages are duplicates, so they should have the same Message-ID. Whether to suppress delivery of duplicates is the choice of an end user, or a system administrator, or a software designer. I don't know which applies to Exchange. But in order to allow the choice, which is correctly a choice to be made at the receiving end, the dups have to be sent with the same Message-ID. I wouldn't say 'blame'. Joseph Brennan Columbia University I T -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman reusing message-id, leads to duplicate message suppression
On Nov 05, 2013, at 02:46 PM, Ralf Hildebrandt wrote: There's not way of turning this off on Exchange Sounds like the same bit of helpfulness that Gmail performs. http://wiki.list.org/x/2IA9 It's a tough problem, but I think Mailman's retention of the original Message-Ids is the right-er approach. Cheers, -Barry signature.asc Description: PGP signature -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman reusing message-id, leads to duplicate message suppression
* Barry Warsaw ba...@python.org: On Nov 05, 2013, at 02:46 PM, Ralf Hildebrandt wrote: There's not way of turning this off on Exchange Sounds like the same bit of helpfulness that Gmail performs. http://wiki.list.org/x/2IA9 It's a tough problem, but I think Mailman's retention of the original Message-Ids is the right-er approach. Strictly speaking, all that womand wanted was to know if the message passed both mailing lists... So she should have more faith :) -- Ralf Hildebrandt Charite Universitätsmedizin Berlin ralf.hildebra...@charite.deCampus Benjamin Franklin http://www.charite.de Hindenburgdamm 30, 12203 Berlin Geschäftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155 -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman reusing message-id, leads to duplicate message suppression
On 11/05/2013 07:14 AM, Ralf Hildebrandt wrote: Strictly speaking, all that womand wanted was to know if the message passed both mailing lists... So she should have more faith :) In cases like this, I suggest that the user check the list's archive. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman reusing message-id, leads to duplicate message suppression
Note that the subject is incorrect. Mailman is not reusing the Message-ID, it is refusing to alter it which is correct behavior according to RFC 5322 (Message-ID is an originator field). I believe that according to RFC 5322 (and predecessors) Mailman SHOULD add a Resent-Message-ID to indicate that it handled the message, but I doubt this would change the duplicate-suppression behavior of Gmail and MS Exchange. Richard Damon writes: This is really a tough problem. For us, there's nothing tough about it. Gmail is the wrong MUA to use if you want to compare mail received by different routes including the null route, otherwise it does what most users want.[1] Microsoft Exchange is not a conformant MTA, no surprise there -- it's not designed to be a conformant MTA. It's intended to be a complete internal communication solution, and where that conflicts with RFC conformance, the RFCs lose. Users will need to find ways to defend themselves against its vagaries -- there's nothing Mailman can do without breaking the world. someone [at Microsoft] may have been confused by Usenet RFCs, and applied them to email, whose system has a lot of similarity (and even share some RFCs), but IS a different system. It's highly unlikely that Microsoft thought past users don't like dealing with duplicates, and in our environment where people send 10MB attachments to ALL, they can actually crash the mail system. Just tell the user that Gmail is not a good MUA if you want to be able to check on sent mail without asking the recipient directly, and that Microsoft products deliberately trade reliable delivery in the Internet sense for what the developers consider to be the convenience of their users. Footnotes: [1] It's surprisingly difficult to handle duplicates truly gracefully. For example, I personally consider the mailing list copy to be the canonical copy of a post, but direct CCs often arrive more quickly. In many cases there has been sufficient delay that the first copy to arrive has already been filed in an archive folder by the time the duplicate arrives. This doesn't make naive duplicate suppression any harder, but it does complicate maintenance of canonical copies in my archive folders. -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman reusing message-id, leads to duplicate message suppression
Ralf Hildebrandt writes: Strictly speaking, all that womand wanted was to know if the message passed both mailing lists... So she should have more faith :) For lists hosted by the same Mailman, Mailman 3 might be able to handle this by adding *both* List-Ids to the header, and only adding the (other) RFC 2369 headers for the list(s) the user is subscribed to. Of course this requires personalization, but that seems to be less and less a problem these days. If that solution is interesting to anybody, please let me know, and I'll check the relevant RFCs to see if multiple values are permissible for List-* headers (and for which ones). Code of course will depend on Barry's permission, but I suppose he wouldn't object to providing it as an option. Also of course, it's only a partial solution, as separate Mailman installations won't be able to coordinate. -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman reusing message-id, leads to duplicate message suppression
On 11/5/13, 6:59 PM, Stephen J. Turnbull wrote: Note that the subject is incorrect. Mailman is not reusing the Message-ID, it is refusing to alter it which is correct behavior according to RFC 5322 (Message-ID is an originator field). I believe that according to RFC 5322 (and predecessors) Mailman SHOULD add a Resent-Message-ID to indicate that it handled the message, but I doubt this would change the duplicate-suppression behavior of Gmail and MS Exchange. It is not clear to me that mailman should add the Resent-* headers. The RFC states: Resent fields SHOULD be added to any message that is reintroduced by a user into the transport system. and later They MUST NOT be used in the normal processing of replies or other such automatic actions on messages. Mailman isn't really a user, and is automatic so I think the second clause is more applicable. -- Richard Damon -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Mailman reusing message-id, leads to duplicate message suppression
Richard Damon writes: It is not clear to me that mailman should add the Resent-* headers. The RFC states: No, it's not clear to me, either. I do have a very strong opinion in favor, to the extent that I would make it an option defaulting to ON. Resent fields SHOULD be added to any message that is reintroduced by a user into the transport system. This is relevant, and I agree that it's not clear whether Mailman is a user as referred to by the RFC. Because the Resent-* headers are purely informational, I think it's harmless to add them: they're the non-MTA equivalent of the trace headers added by MTAs. Mailman is clearly not part of the transport system (it's not an MTA even though does implement parts of SMTP), and often there is human intervention (moderation and user configuration such as notmetoo) in the process of reinserting the message into the transport system. Based on those facts, I think it's reasonable to strengthen the MAY implied by harmless to SHOULD. Also, I believe that Resent-From is preferable to unstandardized mechanisms for detecting loops such as X-Been-There. and later They MUST NOT be used in the normal processing of replies or other such automatic actions on messages. This isn't relevant. The normal processing of replies or other such automatic actions on messages referred to here is by the MUA of the final recipient, *not* the agent that adds the Resent-* headers. It means that replies MUST NOT be directed to any Resent-From address automatically (note there is no Resent-Reply-To), and In-Reply-To and References MUST NOT be initialized based on any Resent-Message-ID. Steve -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
