Re: the right bug severity in case of mbox formats
On 2012-11-29 01:49:55 +0100, Christoph Anton Mitterer wrote: On Wed, 2012-11-28 at 22:06 +, Darren Salt wrote: It would make sense to have that enabled by default, and to ensure that all software in Debian which produces MIME quoted-printable does this, or at least can do this. I agree... but let me add a few notes: 1) Most programs I know of (at least Evolution, haven't checked mutt yet) that do this =46rom encoding do it completely wrong... why? Just quoting (regexp) ^From (.*)$ lines to =46rom \1 is obviously not enough. One also needs to quote ^(*)From (.*)$ lines to =3E\1From \2... otherwise clients could again wrongfully unquote such lines... MUA's should not attempt to remove these quotes *unless* they know there are using mboxrd. So, I don't see any problem with the current behavior. Note: the F encoding may be necessary because mboxo generation corrupts the mail body (and it is done by the sender just in case the recipient uses mboxo, in order to avoid a corruption at this side). But there is no corruption with mboxrd. -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon) -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121129144257.gf5...@xvii.vinc17.org
Re: the right bug severity in case of mbox formats
On 28/11/12 15:34, Darren Salt wrote: Having just viewed the raw text of my message (as sent), there's one other little wrinkle which I already knew but had failed to consider and which makes testing of this useless: gpg handles any 'From ' lines itself in a reversible manner, using '- ' as the prefix. So, so far in this thread we've seen that: * There are certain messages which are losslessly representable in SMTP without using MIME, but not losslessly representable in mboxo format without using MIME; namely, those where a line of the message text starts with From (which become indistinguishable from messages where a line intentionally started with From ). * MUAs that perform GPG clearsigning are mandated to avoid sending such messages (as in: there is an easy and interoperable way not to, and the OpenPGP RFC says they SHOULD use it). Darren's, which appears to be Messenger-Pro, appears to do so without special effort from him. * If you don't restrict yourself to avoiding MIME, any piece of text (or indeed any bytestring) becomes losslessly representable. In particular, MUAs supporting quoted-printable and/or Base64 can avoid sending messages matching /^From / by using MIME and quoted-printable-encoding them. Adam's MUA, which appears to be mutt, does so by using QP, without special effort from him. There are many other classes of message which are not losslessly representable via SMTP without using MIME, including messages not exclusively written in US-ASCII[1], messages where the use of rich text is significant, messages with attachments, messages with very long lines[1], and messages where choice of newline convention is significant. Most people seem to solve this by using MIME, rather than by characterizing SMTP as data corruption. May I suggest the same approach to mboxo? I would personally say that an appropriate severity would go something like this: foo produces/consumes mboxo format: wishlist, potentially wontfix (poor design choice) foo produces/consumes mbox format without documenting whether it means mboxo or mboxrd: minor (documentation bug) foo produces emails formatted for transmission and does not avoid them matching /^From /, e.g. by using QP: minor or perhaps normal, depending on context (interop bug, Postel's principle) foo receives general-purpose email but does not understand MIME, quoted-printable and Base64: important (severe interop bug, Postel's principle, deficient i18n, and applications have already had 20 years to implement MIME) Regards, S [1] may work, or not, depending on the chain of MTAs involved -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/50b66314.5010...@debian.org
Re: the right bug severity in case of mbox formats
* Simon McVittie s...@debian.org, 2012-11-28, 19:16: If you don't restrict yourself to avoiding MIME, any piece of text (or indeed any bytestring) becomes losslessly representable. In particular, MUAs supporting quoted-printable and/or Base64 can avoid sending messages matching /^From / by using MIME and quoted-printable-encoding them. Adam's MUA, which appears to be mutt, does so by using QP, without special effort from him. mutt indeed has an option to that (encode_from), but it's disabled by default. -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121128193148.ga5...@jwilk.net
Re: the right bug severity in case of mbox formats
I demand that Jakub Wilk may or may not have written... * Simon McVittie s...@debian.org, 2012-11-28, 19:16: If you don't restrict yourself to avoiding MIME, any piece of text (or indeed any bytestring) becomes losslessly representable. In particular, MUAs supporting quoted-printable and/or Base64 can avoid sending messages matching /^From / by using MIME and quoted-printable-encoding them. Adam's MUA, which appears to be mutt, does so by using QP, without special effort from him. mutt indeed has an option to that (encode_from), but it's disabled by default. It would make sense to have that enabled by default, and to ensure that all software in Debian which produces MIME quoted-printable does this, or at least can do this. -- | _ | Darren Salt, using Debian GNU/Linux (and Android) | ( ) | | X | ASCII Ribbon campaign against HTML e-mail | / \ | http://www.asciiribbon.org/ From ← should be encoded ☺ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52fb55eaf8%lists...@moreofthesa.me.uk
Re: the right bug severity in case of mbox formats
On Wed, 2012-11-28 at 22:06 +, Darren Salt wrote: It would make sense to have that enabled by default, and to ensure that all software in Debian which produces MIME quoted-printable does this, or at least can do this. I agree... but let me add a few notes: 1) Most programs I know of (at least Evolution, haven't checked mutt yet) that do this =46rom encoding do it completely wrong... why? Just quoting (regexp) ^From (.*)$ lines to =46rom \1 is obviously not enough. One also needs to quote ^(*)From (.*)$ lines to =3E\1From \2... otherwise clients could again wrongfully unquote such lines... For Evolution I've already opened an upstream bug about this (but of course they ignore it like the corruption itself). 2) Be liberal in what you accept and conservative in what you send. I think it's a good idea if we enable this quoting as a safe-guard in all places where we can... but this cannot mean that we can rely on other using it, too. The mail standards never demand it, so we must handle the From_ line corruption by going away from mboxo. Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: the right bug severity in case of mbox formats
On Wed, 2012-11-28 at 19:16 +, Simon McVittie wrote: * There are certain messages which are losslessly representable in SMTP without using MIME, but not losslessly representable in mboxo format without using MIME; yes,... but of course MIME standards neither demand any quoted-printable quoting of From_ lines... so just because a message uses MIME, this doesn't mean one is safe. namely, those where a line of the message text starts with From (which become indistinguishable from messages where a line intentionally started with From ). And of course the more deeply quoted levels (From ...) * MUAs that perform GPG clearsigning are mandated to avoid sending such messages (as in: there is an easy and interoperable way not to, and the OpenPGP RFC says they SHOULD use it). AFAIU, this is only a SHOULD, right? And of course it doesn't help in cases, where one really uses gpg for singing,.. and manually copypastes it to the MUA. May I further remind people, that it's not only signatures that may cause trouble: - People may interpret their mails in scripts,... admittedly this is probably less often the case. - Binary (in the sense: it should not be modified) content like diff output may easily get mangled up. * If you don't restrict yourself to avoiding MIME, any piece of text (or indeed any bytestring) becomes losslessly representable. In particular, MUAs supporting quoted-printable and/or Base64 can avoid sending messages matching /^From / by using MIME and quoted-printable-encoding them. Adam's MUA, which appears to be mutt, does so by using QP, without special effort from him. See my other post where I mention that a) there is no MUST for this kind of quoting and b) most MUAs I've seen so far that do it, do it incompletely. There are many other classes of message which are not losslessly representable via SMTP without using MIME, including messages not exclusively written in US-ASCII[1], messages where the use of rich text is significant, messages with attachments, messages with very long lines[1], and messages where choice of newline convention is significant. Most people seem to solve this by using MIME, rather than by characterizing SMTP as data corruption. I'm not sure whether you can apply the same principle here. When restricting yourself to plain SMTP (without mime and without UT8SMTP) then you have of course a very limited charset. But this doesn't mean already that you get any form of corruption... There only would be such an issue, if your MUA e.g. allows you to enter Unicode characters, mangles them up to 7-bit SMTP but doesn't warn you. Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
