Re: the right bug severity in case of data corruption
On Thu, 2012-11-29 at 08:23 +0100, Paul Gevers wrote: Icedove 10.0.10 (Wheezy, no custom configuration on that front) here. Thunderbird is prone to the issue... and there are only few cases where it doesn't occur... Have a look at https://bugzilla.mozilla.org/show_bug.cgi?id=808450 especially my comment 25, which shows some cases where the issue would not happen with TB. Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: the right bug severity in case of data corruption
On 2012-11-29 01:50:55 +0100, Christoph Anton Mitterer wrote: It's like a serious flaw would have been found in gzip and people would say... oh don't complain... there's already the much better/newer bzip2 or xz. There's a major difference. mbox is buggy by design. Even though mboxrd attempts to fix some problems, there are still MUA's that would show the added in the body (one problem is that they can't detect reliably which mbox format is used). -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon) -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121129140420.gc5...@xvii.vinc17.org
Re: the right bug severity in case of data corruption
On 2012-11-29 01:39:57 +0100, Christoph Anton Mitterer wrote: On Wed, 2012-11-28 at 16:01 +0100, Vincent Lefevre wrote: Even users of mboxo shouldn't even have a problem because in your message the F of the From line is encoded in quoted-printable: | =46rom blahhityblah Fri Jul 8 12:08:34 2011 | From foobarbaz Fri Jul 8 12:08:34 2011 | From quux Fri Jul 8 12:08:34 2011 But this is something that just some friendly MUAs do... it's in no way imposed by any standard to this kind of clever quoting I know, but I was just saying that Adam's test (of the recipients' mail system) was useless because his MUA (Mutt) avoids the problem of having an unencoded From line. -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon) -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121129142424.ge5...@xvii.vinc17.org
Re: the right bug severity in case of data corruption
On 2012-11-29 06:43:06 +, Ian Campbell wrote: On Wed, 2012-11-28 at 16:06 -0500, Nikolaus Rath wrote: Darren Salt lists...@moreofthesa.me.uk writes: (Oops. Failed first time.) Having just viewed the raw text of my message (as sent), there's one other little wrinkle which I already knew but had failed to consider and which makes testing of this useless: gpg handles any 'From ' lines itself in a reversible manner, using '- ' as the prefix. The following SHOULD be 0, 1, and 2 levels of quoting, first to last. From blahhityblah Fri Jul 8 12:08:34 2011 From foobarbaz Fri Jul 8 12:08:34 2011 From quux Fri Jul 8 12:08:34 2011 This is looking different here, Me too Me too. but I am not using any mbox* at all... Me neither, AFAIK. I'm using Exim, with procmail delivering into Maildir and Courier imap to read it. The MISCELLANEOUS section of procmail(1) makes me wonder if this might be procmail's doing. At the very least the conditions where it will do From encoding are too complex for me to grok at this time of day ;-) Definitely neither procmail, nor postfix: I've sent a mail to myself with a From (not using QP -- checked that) at the beginning of the line, and everything was fine. The problem may come from the mailing-list software. -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon) -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121129152924.gj5...@xvii.vinc17.org
Re: the right bug severity in case of data corruption
On Wed, Nov 28, 2012 at 07:52:16AM +0100, Bernhard R. Link wrote: * Adam Borowski kilob...@angband.pl [121127 16:32]: So, what's the reason mbox is still the default in Debian? Because it works and causes the smallest amount of problems given all the other changes. Like, locking issues, multiple ways to corrupt mbox files (possibly resulting in loss of all mail in the directory), having to read/write multiple gigabytes at 80 MB/s every operation, etc ... Among other gains, data loss because of mboxo would be gone. Continuing to call that data loss makes it quite hard to take anything else you say seriously. Continuing to call breaking gpg signatures harmless makes it quite hard to take anything else you say seriously. Especially that multiple solutions (mboxrd, Maildir) exist. Let's give it a test, this mail should be signed: From blahhityblah Fri Jul 8 12:08:34 2011 From foobarbaz Fri Jul 8 12:08:34 2011 From quux Fri Jul 8 12:08:34 2011 If the signature is invalid, your setup is broken. -- How to squander your resources: those silly Swedes have a sauce named hovmästarsås, the best thing ever to put on cheese, yet they waste it solely on mere salmon. signature.asc Description: Digital signature
Re: the right bug severity in case of data corruption
On 2012-11-28 11:47:38 +0100, Adam Borowski wrote: Let's give it a test, this mail should be signed: From blahhityblah Fri Jul 8 12:08:34 2011 From foobarbaz Fri Jul 8 12:08:34 2011 From quux Fri Jul 8 12:08:34 2011 If the signature is invalid, your setup is broken. Even users of mboxo shouldn't even have a problem because in your message the F of the From line is encoded in quoted-printable: | =46rom blahhityblah Fri Jul 8 12:08:34 2011 | From foobarbaz Fri Jul 8 12:08:34 2011 | From quux Fri Jul 8 12:08:34 2011 So, there's no possible corruption due to the mailbox format. -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon) -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121128150113.gb5...@xvii.vinc17.org
Re: the right bug severity in case of data corruption
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I demand that Vincent Lefevre may or may not have written... On 2012-11-28 11:47:38 +0100, Adam Borowski wrote: Let's give it a test, this mail should be signed: From blahhityblah Fri Jul 8 12:08:34 2011 From foobarbaz Fri Jul 8 12:08:34 2011 From quux Fri Jul 8 12:08:34 2011 If the signature is invalid, your setup is broken. Even users of mboxo shouldn't even have a problem because in your message the F of the From line is encoded in quoted-printable: | =46rom blahhityblah Fri Jul 8 12:08:34 2011 | From foobarbaz Fri Jul 8 12:08:34 2011 | From quux Fri Jul 8 12:08:34 2011 So, there's no possible corruption due to the mailbox format. Let's see if this can be done properly... if this fails, I'll use raw SMTP. Of course, this is subject to MXes and list servers not modifying the content...! - From blahhityblah Fri Jul 8 12:08:34 2011 From foobarbaz Fri Jul 8 12:08:34 2011 From quux Fri Jul 8 12:08:34 2011 - -- | _ | Darren Salt, using Debian GNU/Linux (and Android) | ( ) | | X | ASCII Ribbon campaign against HTML e-mail | / \ | http://www.asciiribbon.org/ I'd like to, but my favourite advert is on TV. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iD8DBQFQtimhzbwfTn7RbcARAsLsAKCqjIOvgqHc+q12po8wtOMkY9HKhgCfQDpt D56zQdIVsEAzF+4xb8TW82I= =Zn6Y -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52fb2f8ea8%lists...@moreofthesa.me.uk
Re: the right bug severity in case of data corruption
Having just viewed the raw text of my message (as sent), there's one other little wrinkle which I already knew but had failed to consider and which makes testing of this useless â gpg handles any âÿóÿýFrom â lines itself in a -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52fb300cd4%lists...@moreofthesa.me.uk
Re: the right bug severity in case of data corruption
(Oops. Failed first time.) Having just viewed the raw text of my message (as sent), there's one other little wrinkle which I already knew but had failed to consider and which makes testing of this useless: gpg handles any 'From ' lines itself in a reversible manner, using '- ' as the prefix. The following SHOULD be 0, 1, and 2 levels of quoting, first to last. From blahhityblah Fri Jul 8 12:08:34 2011 From foobarbaz Fri Jul 8 12:08:34 2011 From quux Fri Jul 8 12:08:34 2011 -- | _ | Darren Salt, using Debian GNU/Linux (and Android) | ( ) | | X | ASCII Ribbon campaign against HTML e-mail | / \ | http://www.asciiribbon.org/ For every action, there is an equal and opposite criticism. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52fb320390%lists...@moreofthesa.me.uk
Re: the right bug severity in case of data corruption
On Wed, Nov 28, 2012 at 03:34:32PM +, Darren Salt wrote: Having just viewed the raw text of my message (as sent), there's one other little wrinkle which I already knew but had failed to consider and which makes testing of this useless: gpg handles any 'From ' lines itself in a reversible manner, using '- ' as the prefix. Definitely reversible? How does it distinguish 'From ' from '- From' prior to signing? Ad infinitum down the rabbit hole… The following SHOULD be 0, 1, and 2 levels of quoting, first to last. It was for me (Maildir) -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121128172920.GB8248@debian
Re: the right bug severity in case of data corruption
* Jon Dowland j...@debian.org, 2012-11-28, 17:29: Having just viewed the raw text of my message (as sent), there's one other little wrinkle which I already knew but had failed to consider and which makes testing of this useless: gpg handles any 'From ' lines itself in a reversible manner, using '- ' as the prefix. Definitely reversible? How does it distinguish 'From ' from '- From' prior to signing? Ad infinitum down the rabbit hole… http://tools.ietf.org/html/rfc4880#section-7.1 Dash-escaped cleartext is the ordinary cleartext where every line starting with a dash '-' (0x2D) is prefixed by the sequence dash '-' (0x2D) and space ' ' (0x20). This prevents the parser from recognizing armor headers of the cleartext itself. An implementation MAY dash-escape any line, SHOULD dash-escape lines commencing From followed by a space, and MUST dash-escape any line commencing in a dash. The message digest is computed using the cleartext itself, not the dash-escaped form. -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121128181549.ga1...@jwilk.net
Re: the right bug severity in case of data corruption
The following SHOULD be 0, 1, and 2 levels of quoting, first to last. From blahhityblah Fri Jul 8 12:08:34 2011 From foobarbaz Fri Jul 8 12:08:34 2011 From quux Fri Jul 8 12:08:34 2011 So if I understand correctly, I now identified my e-mail provider as using mboxo? I indeed got 1, 1 and 2 levels of quoting. Paul signature.asc Description: OpenPGP digital signature
Re: the right bug severity in case of data corruption
Darren Salt lists...@moreofthesa.me.uk writes: (Oops. Failed first time.) Having just viewed the raw text of my message (as sent), there's one other little wrinkle which I already knew but had failed to consider and which makes testing of this useless: gpg handles any 'From ' lines itself in a reversible manner, using '- ' as the prefix. The following SHOULD be 0, 1, and 2 levels of quoting, first to last. From blahhityblah Fri Jul 8 12:08:34 2011 From foobarbaz Fri Jul 8 12:08:34 2011 From quux Fri Jul 8 12:08:34 2011 This is looking different here, but I am not using any mbox* at all... Are you sure you sent it with 0 quotes? Best, -Nikolaus -- »Time flies like an arrow, fruit flies like a Banana.« PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87txs9igi6@inspiron.ap.columbia.edu
Re: the right bug severity in case of data corruption
On Wed, Nov 28, 2012 at 03:34:32PM +, Darren Salt wrote: (Oops. Failed first time.) Having just viewed the raw text of my message (as sent), there's one other little wrinkle which I already knew but had failed to consider and which makes testing of this useless: gpg handles any 'From ' lines itself in a reversible manner, using '- ' as the prefix. The following SHOULD be 0, 1, and 2 levels of quoting, first to last. From blahhityblah Fri Jul 8 12:08:34 2011 From foobarbaz Fri Jul 8 12:08:34 2011 From quux Fri Jul 8 12:08:34 2011 I think it was a good idea to sign the message and ask to check the signature. -- WBR, wRAR signature.asc Description: Digital signature
Re: the right bug severity in case of data corruption
I demand that Nikolaus Rath may or may not have written... Darren Salt lists...@moreofthesa.me.uk writes: [snip] The following SHOULD be 0, 1, and 2 levels of quoting, first to last. From blahhityblah Fri Jul 8 12:08:34 2011 From foobarbaz Fri Jul 8 12:08:34 2011 From quux Fri Jul 8 12:08:34 2011 This is looking different here, but I am not using any mbox* at all... Are you sure you sent it with 0 quotes? Yes, and I saw it arrive back here *with* extra quoting. Re-sending it locally to myself (via my MTA) works fine; it comes back unquoted. Could it be that the postfix installation on bendel.d.o is using mboxo? -- | _ | Darren Salt, using Debian GNU/Linux (and Android) | ( ) | | X | ASCII Ribbon campaign against HTML e-mail | / \ | http://www.asciiribbon.org/ The reader of this tagline only exists while you are reading it. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52fb538b28%lists...@moreofthesa.me.uk
Re: the right bug severity in case of data corruption
On Wed, 2012-11-28 at 16:01 +0100, Vincent Lefevre wrote: Even users of mboxo shouldn't even have a problem because in your message the F of the From line is encoded in quoted-printable: | =46rom blahhityblah Fri Jul 8 12:08:34 2011 | From foobarbaz Fri Jul 8 12:08:34 2011 | From quux Fri Jul 8 12:08:34 2011 But this is something that just some friendly MUAs do... it's in no way imposed by any standard to this kind of clever quoting So, there's no possible corruption due to the mailbox format. Not fully true either... well it depends on your MUA/tool, because some are buggy in a way that they'll unquote his: From foobarbaz... line which of course mustn't happen. Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: the right bug severity in case of data corruption
On Wed, 2012-11-28 at 19:55 +0100, Paul Gevers wrote: So if I understand correctly, I now identified my e-mail provider as using mboxo? I indeed got 1, 1 and 2 levels of quoting. Depends... were you using his webmail? Then probably yes... But it could have also been your local MUA (when you were using one) I also get levels 1, 1, 2... in my case an old Evolution 2.32.x being the bad guy. Chris. smime.p7s Description: S/MIME cryptographic signature
Re: the right bug severity in case of data corruption
On Tue, 2012-11-27 at 16:32 +0100, Adam Borowski wrote: So, what's the reason mbox is still the default in Debian? Among other gains, data loss because of mboxo would be gone. Just posted some reasons[0] (but as I see now, some of them have already been named by others... But in general... I think whether a better alternative is available, and whether Debian should switch to it, doesn't matter at all here: These programs still offer mbox, even if we decided to change all defaults to maildir... people could still shoot themselves into their feet (without their own fault). It's like a serious flaw would have been found in gzip and people would say... oh don't complain... there's already the much better/newer bzip2 or xz. Further, e.g. for Thunderbird there IS currently NO way to avoid mbox... even when having e.g. a IMAP server locally that actually stores your mail, TB seems to temporary store mails as mbox. [3]. Out of a habit, I guess. With current disk sizes, no one should care about a few gigs here, a few gigs there. Unless you need to read a mbox linearly, that is. I guess you've never run a really really large mail archive, have you? Cheers, Chris. [0] http://lists.debian.org/debian-devel/2012/11/msg00853.html smime.p7s Description: S/MIME cryptographic signature
Re: the right bug severity in case of data corruption
On Wed, 2012-11-28 at 16:06 -0500, Nikolaus Rath wrote: Darren Salt lists...@moreofthesa.me.uk writes: (Oops. Failed first time.) Having just viewed the raw text of my message (as sent), there's one other little wrinkle which I already knew but had failed to consider and which makes testing of this useless: gpg handles any 'From ' lines itself in a reversible manner, using '- ' as the prefix. The following SHOULD be 0, 1, and 2 levels of quoting, first to last. From blahhityblah Fri Jul 8 12:08:34 2011 From foobarbaz Fri Jul 8 12:08:34 2011 From quux Fri Jul 8 12:08:34 2011 This is looking different here, Me too but I am not using any mbox* at all... Me neither, AFAIK. I'm using Exim, with procmail delivering into Maildir and Courier imap to read it. The MISCELLANEOUS section of procmail(1) makes me wonder if this might be procmail's doing. At the very least the conditions where it will do From encoding are too complex for me to grok at this time of day ;-) Ian. -- Ian Campbell QOTD: I've always wanted to work in the Federal Mint. And then go on strike. To make less money. signature.asc Description: This is a digitally signed message part
Re: the right bug severity in case of data corruption
On 29-11-12 01:43, Christoph Anton Mitterer wrote: On Wed, 2012-11-28 at 19:55 +0100, Paul Gevers wrote: So if I understand correctly, I now identified my e-mail provider as using mboxo? I indeed got 1, 1 and 2 levels of quoting. Depends... were you using his webmail? Then probably yes... But it could have also been your local MUA (when you were using one) Well, both show the same (I already checked myself, thinking along the same lines). But I think others in this thread have suspicion that it might also be caused by an intermediate server. I also get levels 1, 1, 2... in my case an old Evolution 2.32.x being the bad guy. Icedove 10.0.10 (Wheezy, no custom configuration on that front) here. Paul signature.asc Description: OpenPGP digital signature
Re: the right bug severity in case of data corruption
On Wed, Nov 28, 2012 at 05:29:20PM +, Jon Dowland wrote: The following SHOULD be 0, 1, and 2 levels of quoting, first to last. It was for me (Maildir) Just rechecked, I'm wrong - the first line was quoted. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121129075041.GB24783@debian
Re: the right bug severity in case of data corruption
For anyone else following along at home who is slightly puzzled by all this, http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/mail-mbox-formats.html explains the different mbox formats, what 'From_' means, etc. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121127115625.GB8359@debian
Re: the right bug severity in case of data corruption
On Tue, Nov 27, 2012 at 11:56:25AM +, Jon Dowland wrote: For anyone else following along at home who is slightly puzzled by all this, http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/mail-mbox-formats.html explains the different mbox formats, what 'From_' means, etc. Quoting from that page: # With the advent and now widespread adoption of the superior Maildir format # over the past several years, the entire mbox family of mailbox formats # is gradually becoming irrelevant, and of only historical interest. which is no news. And you can't really run a mail server in mbox if you ever receive mail from business users: for them, sending the text as an image wrapped in a Word document is the rule rather than an exception[1]. With mbox, every access requires linearly scanning the whole file. Users tend to keep loads of junk so you can expect multi-GB[2] mboxes. There are two typical cases for mta installations: * a real mail server: you need to be able to handle large mails * no mail other than cron/etc: storage type is irrelevant So, what's the reason mbox is still the default in Debian? Among other gains, data loss because of mboxo would be gone. [1]. Bonus points for printing out your plain text piece of mail, highlighting something with a marker, adding comments in pen, scanning it back and then mailing. [2]. Let's take a look at one of my users' ~/Maildir. I whine about too much junk from time to time[3], asking to delete at least pointless attachments, so the two biggest dirs are only 2.8GB and 2.3GB, both around 6.5k pieces, the largest of which is 48MB. [3]. Out of a habit, I guess. With current disk sizes, no one should care about a few gigs here, a few gigs there. Unless you need to read a mbox linearly, that is. -- How to squander your resources: those silly Swedes have a sauce named hovmästarsås, the best thing ever to put on cheese, yet they waste it solely on mere salmon. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121127153215.ga26...@angband.pl
Re: the right bug severity in case of data corruption
* Adam Borowski kilob...@angband.pl [121127 16:32]: So, what's the reason mbox is still the default in Debian? Because it works and causes the smallest amount of problems given all the other changes. Among other gains, data loss because of mboxo would be gone. Continuing to call that data loss makes it quite hard to take anything else you say seriously. Bernhard R. Link -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121128065215.ga14...@client.brlink.eu
Re: the right bug severity in case of data corruption
Hey Henrique. On Sat, 2012-11-24 at 22:27 -0200, Henrique de Moraes Holschuh wrote: ... ?? At least for postfix, I'd expect them to accept a patch for local(8) to not quote From lines as a config option, First, I've never asked not to quote From_ lines at all, cause this would really lead to massive mail corruption / phantom messages. What I usually proposed was mboxrd, which quotes _all_ From_ lines and not just some. with the default being to keep the current behaviour (i.e. quote them). But apart from this,... yes you're right, Wietse already said he would probably accept such a patch, but with the default still exposing the users to data corruption. Which does of course not necessarily mean that anyone is writing such a patch, though. Look, if it were severity critical, given how long this situation stands, we'd not have this thread and nobody would be doing this. Well or just no one ever recognised it, or those who did decided for themselves, they could live with it... In postfix' case, it is documented like all heck, for example (see local(8) manpage) Quote? Actually I had talked about just that lack with Wietse, too, and he didn't complain when I mentioned that there was no big fat hint in local(8) about mboxo being used and it's vulnerability to corruption. and it can trivially be configured to use something else to deliver mail to mbox files (such as procmail, etc). Of course, but from the postfix documentation, nothing AFAICS tells one the need to use a 3rd party MDA, because postfix' own were broken. We [Debian] could deprecate anything that uses mboxo (mbox old) format as the native/preferred storage format, and configure our stuff where possible to never use mboxo by default. That's about it. That's what I've meant,... and therefore I suggested to add notice about the issue in those places where the user are most likely to read it, e.g. package description, README.Debian and depending how prone a program is to it, perhaps even via some hack like a informing debconf dialogue (though someone has indicated this might conflict the policy). E.g. Thunderbird is so much more vulnerable to it (right now) than Evolution (where at least current versions rarely use mbox, unless you import or export mail)... so for it I'd have said the inevitable debconf warning would be appropriate. Further my suggestion was, to inform users (typically via NEWS.Debian) even when the issue has been fixed already, like in getmail, what has happened over years. It's like when you have some scientific application which does wrong calculations... it's not fair to just fix it and think everything's fine now... one also has to tell people that all their previous data is likely wrong. btw: I've always thought it meant mbox original ;-) But first, you need to get support for better mbox formats on the important stuff. Well I guess you'll agree that this is not that easy and usually a duty for upstream. I personally, may try to invest some time into postfix, but after my struggles with the Evolution and TB upstreams, I have little interest to - sorry for the rude tone - fix their crap. Especially as it seems, well at least for TB, that mbox import/export is coded quite crude in awkwardly many places... We [users/developers] can write patches to teach important software to use something less retarded than mboxo by default, and pester upstream to take them. Just complaining about it obviously won't help, since people don't see it as a problem at all. You need to do _all_ the leg work, write the new functionality, and test the heck out of it before you can really expect upstream to accept the change. I think you know from your own experience, that one usually already has a big stack of projects where one is involved in... and that changes to core places of bigger projects like Evolution/TB are usually not just an hour of coding. If not, than the majority opinion might perhaps even convince the maintainers to handle this with some higher severity :) That's not how it works. Someone needs to come up with *well tested* patches for everything worth fixing. THEN you can ask for some sort of Debian-wide pressure to get rid of mboxo outside of extra/old-libs... Again, I've never asked from the Debian maintainers to write patches... My point was that we need reasonable warnings for our users, such as mentioned above. I guess such warnings are very easy to add for any maintainer, right? And as long as those are not in place, my point was, that a high severity is appropriate to give at least users with the widespread apt-listbugs a chance to notice what's going on. Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
the right bug severity in case of data corruption
Hi. I've recently reported several bugs against MUAs and mail tools, that employ the mboxo (note the trailing o) format to either store or import mails. This however leads to irrecoverable data corruption, as the partial quoting of so called From_ lines cannot be undone anymore. An easy solution for that dilemma is known for years, namely the other mbox formats (either mboxrd, mboxcl or mboxcl2), of which mboxrd is typically the one which fits best the needs for staying backwards compatible. Given that mails are the core business and data of MUAs and mail tools, I'd personally say that any corruption of them, even if it seems to be minor deserves the most critical severity. Similarly, if a DBMS like postgres would silently modify integers, even if joust a tiny bit, it would be considered unacceptable. For mails one may easily think, that a small corruption isn't a problem, but not only does it break signatures or perhaps corrupt in-line content like patches, it's IMHO also not the decision of upstream, the maintainers or anyone else but each single user which kind of corruption of his data he/she considers to be severe. No first I've had reported these corruptions at the different upstreams (and apart from KMail and mutt...all the major players I've tested, e.g. Thunderbird, Evolution, getmail, postfix, were affected). Apart from the getmail upstream all reacted rather stubborn and without much insight,... bringing up obscure arguments like this corruption has always been the case historically, therefore it's ok. Well we, as Debian, can't of course force upstreams to fix their crap, but IMHO neither should we let our users at risk for silent data corruption. So I've opened bugs at the BTS, too, with severities critical (justification: data loss) where I mentioned the upstream bug and further suggested what I think we should do at the Debian level to adequately warn users. IIRC, in all but the getmail case (where the issue has been fixed) upstream, this was neither accepted by the Debian maintainers, referring to the same obscure arguments from the upstreams. I personally have no longer much interest in tracing this family of issues, especially when being confronted with so much narrow-mindedness and arrogance (IMHO, deciding for the users that they can live with mail corruption is arrogance)...and when it's tried at nearly all levels to hide these corruption bugs away behind duplicates or lesser severities.. So,... bringing this up here at d-d, as I think it would be good for Debian to have a well thought position in how to handle this family of corruption bugs... If it's agreed upon with upstreams/maintainers that it's ok to let people live with them... fine for me... If not, than the majority opinion might perhaps even convince the maintainers to handle this with some higher severity :) Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: the right bug severity in case of data corruption
* Christoph Anton Mitterer cales...@scientia.net [121124 17:42]: I've recently reported several bugs against MUAs and mail tools, that employ the mboxo (note the trailing o) format to either store or import mails. So,... bringing this up here at d-d, as I think it would be good for Debian to have a well thought position in how to handle this family of corruption bugs... I'd say if you complain about a tool not documenting what mbox format it is using, that is a minor bug. If you want an option to also support another mbox format but mboxo, then I'd vote for wishlist severity. Ambiquity about lines starting with from in mboxo format is the same like storing the value 0007 in an integer and getting 7 back when asking for the value. Or like storing a filename in a FAT filesystem and getting it back when asking for a file with the name converted to upper case. Bernhard R. Link -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121124201101.gb7...@client.brlink.eu
Re: the right bug severity in case of data corruption
On Sat, 24 Nov 2012, Christoph Anton Mitterer wrote: This however leads to irrecoverable data corruption, as the partial quoting of so called From_ lines cannot be undone anymore. An easy solution for that dilemma is known for years, namely the other mbox formats (either mboxrd, mboxcl or mboxcl2), of which mboxrd is typically the one which fits best the needs for staying backwards compatible. ... No first I've had reported these corruptions at the different upstreams (and apart from KMail and mutt...all the major players I've tested, e.g. Thunderbird, Evolution, getmail, postfix, were affected). ... Apart from the getmail upstream all reacted rather stubborn and without much insight,... bringing up obscure arguments like this corruption has always been the case historically, therefore it's ok. At least for postfix, I'd expect them to accept a patch for local(8) to not quote From lines as a config option, with the default being to keep the current behaviour (i.e. quote them). Well we, as Debian, can't of course force upstreams to fix their crap, but IMHO neither should we let our users at risk for silent data corruption. So I've opened bugs at the BTS, too, with severities critical Look, if it were severity critical, given how long this situation stands, we'd not have this thread and nobody would be doing this. It is severity important or normal, I'd say. In postfix' case, it is documented like all heck, for example (see local(8) manpage), and it can trivially be configured to use something else to deliver mail to mbox files (such as procmail, etc). So,... bringing this up here at d-d, as I think it would be good for Debian to have a well thought position in how to handle this family of corruption bugs... We [Debian] could deprecate anything that uses mboxo (mbox old) format as the native/preferred storage format, and configure our stuff where possible to never use mboxo by default. That's about it. But first, you need to get support for better mbox formats on the important stuff. We [users/developers] can write patches to teach important software to use something less retarded than mboxo by default, and pester upstream to take them. Just complaining about it obviously won't help, since people don't see it as a problem at all. You need to do _all_ the leg work, write the new functionality, and test the heck out of it before you can really expect upstream to accept the change. If not, than the majority opinion might perhaps even convince the maintainers to handle this with some higher severity :) That's not how it works. Someone needs to come up with *well tested* patches for everything worth fixing. THEN you can ask for some sort of Debian-wide pressure to get rid of mboxo outside of extra/old-libs... -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20121125002747.ga17...@khazad-dum.debian.net
