Bug#109171: Use Maildir format by default
Raul == Raul Miller [EMAIL PROTECTED] writes: Raul Also note that mbox format need not be lossy -- MDA should Raul quote with any line matching regexp ^*From , MUA should Raul always remove one level of quoting at display time. Any Raul software not following this is losing information (and Raul therefore buggy). On Thu, Sep 06, 2001 at 12:20:49PM +1000, Brian May wrote: What is suppose to happen to signed messages? Current practise, I believe is to quote the From headers before the message is signed, in which case the quoting cannot be removed without damaging the signature. From my point of view, that's completely broken. From my ~/.gnupg/options file: # Because some mailers change lines starting with From to From # it is good to handle such lines in a special way when creating # cleartext signatures; all other PGP versions it this way too. Use quoted-printable. # To enable full OpenPGP compliance you have to remove this option. That too. escape-from-lines which is really stupid IMHO, as it means the message gets converted for mbox format before it is even sent. Yep. And there are existing MDAs which will quote From, yielding From, and quote From yielding From, etc. I happen to be using one of them. For fun, I'm going to pgp sign this message... [A quick test shows that mutt is broken if I use an MDA which does multiple From line quoting. Time to file a bug report.] -- Raul pgpARj7Q5DBqp.pgp Description: PGP signature
Bug#109171: Use Maildir format by default
Raul == Raul Miller [EMAIL PROTECTED] writes: Raul Use quoted-printable. Ok, I didn't realize that quoted printable coped with this. My experimentation shows that a line starting with From is turned into =46rom (using mutt). Not very readable to a human reader, but at least any program that understands quoted-printable (including mutt) can display it properly. This was GnuPG signed, with OpenPGP non-compliant options removed, and the signature came out looking OK. -- Brian May [EMAIL PROTECTED]
Bug#109171: Use Maildir format by default
[I know this thread is old.] On Sun, Aug 19, 2001 at 08:31:58AM -0700, Aaron Lehmann wrote: I'm interested in performance differences. A big flaw of the mbox format is that every byte of the file must be read to extract headers of the messages .. i.e. display a list of the messages without necessarily showing any data. Does maildir employ a summarization technique or must an MUA open every single file to extract the mailbox headers? No. Also note that mbox format need not be lossy -- MDA should quote with any line matching regexp ^*From , MUA should always remove one level of quoting at display time. Any software not following this is losing information (and therefore buggy). Also, for reference: __ Date: 8 Jan 1997 01:15:39 - Message-ID: [EMAIL PROTECTED] From: D. J. Bernstein [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: Re: wishlist -- shared Maildir/ Therefore, why not expand the Maildir definition to support the issues that an MUA has as well? It's already adequate for most MUAs, with info serving the same role for maildir as Status serves for mbox. (Yes, agreement between MUAs on the info format would be nice. Yes, I will coordinate.) It's not adequate for MH-style sequences. Handling sequences reliably over NFS is exceedingly difficult; I certainly wouldn't want to try. As a practical matter, I don't see one-file-per-message as a good format for long-term message archiving. It's bitten by all of the filesystem's traditional performance problems. If I were writing an MUA, I'd want an archive format designed for low space, very fast lookup, and fast append: e.g., each message gzipped individually, concatenated into a single file, with a separate random-access index listing positions and sizes. ---Dan __ I think the koobera.math.uic.edu address is still good, but his cannonical address is [EMAIL PROTECTED] -- Raul
Bug#109171: Use Maildir format by default
Raul == Raul Miller [EMAIL PROTECTED] writes: Raul [I know this thread is old.] Raul On Sun, Aug 19, 2001 at 08:31:58AM -0700, Aaron Lehmann wrote: I'm interested in performance differences. A big flaw of the mbox format is that every byte of the file must be read to extract headers of the messages .. i.e. display a list of the messages without necessarily showing any data. Does maildir employ a summarization technique or must an MUA open every single file to extract the mailbox headers? Raul No. Raul Also note that mbox format need not be lossy -- MDA should Raul quote with any line matching regexp ^*From , MUA should Raul always remove one level of quoting at display time. Any Raul software not following this is losing information (and Raul therefore buggy). What is suppose to happen to signed messages? Current practise, I believe is to quote the From headers before the message is signed, in which case the quoting cannot be removed without damaging the signature. From my ~/.gnupg/options file: # Because some mailers change lines starting with From to From # it is good to handle such lines in a special way when creating # cleartext signatures; all other PGP versions it this way too. # To enable full OpenPGP compliance you have to remove this option. escape-from-lines which is really stupid IMHO, as it means the message gets converted for mbox format before it is even sent. -- Brian May [EMAIL PROTECTED]
Bug#109171: Use Maildir format by default
Hi, I am not sure this is a good timing for this to be a policy proposal. Please read on to see why I think so. Cesar == Cesar Eduardo Barros [EMAIL PROTECTED] writes: Cesar Debian uses the traditional mailbox style by default. [ Snip reasons for why this is not a good idea ] Cesar My proposal is to: Cesar 1. Use the maildir format by default for storage of the user's Cesarincoming mail. This means both the MTA must be configured Cesarfor maildir by default (e.g. in eximconfig) and all the CesarMUAs must be able to grok it without needing any manual Cesarconfiguration This is not a policy related proposal -- until we are sure that the MUA's can handle the Maildir format, and the MTA's can be configured to use it as well, this proposal is unlikely to fly. This is not something that can be accomplished by merely changing policy -- and indeed, you do not need to change policy in order to get this accomplished in the first case. Once we have the MTA's and the MUA's changed to be able to be configured to use the MAILDIR format, we can go ahead and formulate the policy for it. I am not very familiar with the status of the various MUA's out there in Debian, so I am not sure how close we are. Cesar 2. Use the maildir format to store the user folders by default Cesarif the user's MUA allows it The users mail is spooled by the MTA (using the MDA). How is the MDA supposed to know if the users MUA of the week is able to grok this format? Cesar The issues are: Cesar - Debian is about choice. So, of course mailbox vs maildir Cesar will end up being a configuration question. Just make Cesar maildir the default, please. Well, as long as we are sure that the various mail related subsystems on Debian can actually do this. Are we? Cesar - Standards for where to put the user's incoming mail/where in Cesar $HOME to put the folders. I believe most programs which use Cesar maildir already share common places, but putting them in Cesar policy would be good. Well, documenting standard practice whould not be a bad idea. Cesar - Making programs which can't use it yet be able to use it This has to be done before we mandate it in policy. Cesar - Somehow making programs able to guess which is the mbox/mdir Cesar choice of the day (perhaps using debconf). Again, this has to be done before it can be mandated in a policy proposal. Cesar - Dealing with indecise people who change the default while Cesar the system is live I think these issues need to have a consensus solution before we can go on with a formal proposal/amendment manoj -- For courage mounteth with occasion. William Shakespeare, King John Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/%7Esrivasta/ 1024R/C7261095 print CB D9 F4 12 68 07 E4 05 CC 2D 27 12 1D F5 E8 6E 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C
Bug#109171: Use Maildir format by default
On Sat, Aug 18, 2001 at 08:36:05PM -0300, Cesar Eduardo Barros wrote: Package: debian-policy Version: 3.5.6.0 Severity: wishlist Debian uses the traditional mailbox style by default. However, it has some disadvantages over maildir -- one of them is that it does a non-reversible modification to the message's text if it contains the sequence 'From ' at the beginning of a line. MUAs like mutt also save the user's mail folders in the same problematic format. well, FWIW, one of the first actions I do on newly installed debian systems is to change mail delivery to maildirs My proposal is to: 1. Use the maildir format by default for storage of the user's incoming mail. This means both the MTA must be configured for maildir by default (e.g. in eximconfig) and all the MUAs must be able to grok it without needing any manual configuration elm (so far still very popular) does not work with maildirs. 2. Use the maildir format to store the user folders by default if the user's MUA allows it The issues are: - Debian is about choice. So, of course mailbox vs maildir will end up being a configuration question. Just make maildir the default, please. does not matter much when it is a configuration question :-) - Standards for where to put the user's incoming mail/where in $HOME to put the folders. I believe most programs which use maildir already share common places, but putting them in policy would be good. - Making programs which can't use it yet be able to use it - Somehow making programs able to guess which is the mbox/mdir choice of the day (perhaps using debconf). environmental variable MAIL - Dealing with indecise people who change the default while the system is live that's the biggest problem. Changing the setup on live system is not trivial to automatize. I would just go for exim postinst configuration question, and if somebody wants to switch his system over, he can do it manually -- --- | Radovan Garabik http://melkor.dnp.fmph.uniba.sk/~garabik/ | | __..--^^^--..__garabik @ melkor.dnp.fmph.uniba.sk | --- Antivirus alert: file .signature infected by signature virus. Hi! I'm a signature virus! Copy me into your signature file to help me spread!
Bug#109171: Use Maildir format by default
On Aug 19, Cesar Eduardo Barros [EMAIL PROTECTED] wrote: eximconfig) and all the MUAs must be able to grok it without needing any manual configuration Are you offering to be the one which will patch everything and make the upstream maintainers merge maildir support? And don't forget POP servers too. - Debian is about choice. So, of course mailbox vs maildir will end up being a configuration question. Just make maildir the default, please. No fucking way. - Standards for where to put the user's incoming mail/where in $HOME to put the folders. I believe most programs which use maildir already share common places, but putting them in policy would be good. Looks like a good idea. - Making programs which can't use it yet be able to use it You can do this even without changing policy, you just have to start coding. 2. Benchmarking mbox versus maildir http://www.courier-mta.org/mbox-vs-maildir/ This has to be a joke. Everybody who knows something about mail servers can tell UW-* programs suck. The benchmark only confirms this, not that maildir is faster (it usually is not, and it definitely is not for tipical POP servers). It also does not discuss the effects of caching in real life. insert standard Debian should take the lead, look at the xterm backspace mess and how well Debian handled it, etc Debian should make sensible decisions about use of available technology. The reality is that maildir is only useful on some setups with remote mounted mailboxes and broken NFS locking. -- ciao, Marco
Bug#109171: Use Maildir format by default
In article [EMAIL PROTECTED], Marco d'Itri [EMAIL PROTECTED] wrote: 2. Benchmarking mbox versus maildir http://www.courier-mta.org/mbox-vs-maildir/ This has to be a joke. Everybody who knows something about mail servers can tell UW-* programs suck. The benchmark only confirms this, not that maildir is faster True. (it usually is not, and it definitely is not for tipical POP servers). Still, deleting messages from a mbox-style mailbox means copying the entire mailbox usually at least twice. That can cause a heavy load on your mailserver. Also it is nessecary to scan the entire mbox at startup - readdir() is usually faster (if you skip the stat()) Debian should make sensible decisions about use of available technology. The reality is that maildir is only useful on some setups with remote mounted mailboxes and broken NFS locking. There certainly are mailbox formats that are better than maildir, but IMHO unix-style mbox files isn't one of them. Mike.
Bug#109171: Use Maildir format by default
On Sun, Aug 19, 2001 at 12:51:05PM +, Miquel van Smoorenburg wrote: (it usually is not, and it definitely is not for tipical POP servers). Still, deleting messages from a mbox-style mailbox means copying the entire mailbox usually at least twice. Copying it once and renaming it, surely? Maildir is also a good way to go if you want to have mail shared across two machines that aren't always connected. (Using unison.deb or similar, eg) Debian should make sensible decisions about use of available technology. The reality is that maildir is only useful on some setups with remote mounted mailboxes and broken NFS locking. There certainly are mailbox formats that are better than maildir, but IMHO unix-style mbox files isn't one of them. There are? Any pointers? Cheers, aj, who doesn't really think there's much value in changing the default mailbox format for Debian -- Anthony Towns [EMAIL PROTECTED] http://azure.humbug.org.au/~aj/ I don't speak for anyone save myself. GPG signed mail preferred. ``_Any_ increase in interface difficulty, in exchange for a benefit you do not understand, cannot perceive, or don't care about, is too much.'' -- John S. Novak, III (The Humblest Man on the Net) pgpBHozvefJs2.pgp Description: PGP signature
Bug#109171: Use Maildir format by default
On Sun, Aug 19, 2001 at 01:32:12PM +0200, Marco d'Itri wrote: and broken NFS locking. Is there any other kind? -- G. Branden Robinson| When dogma enters the brain, all Debian GNU/Linux | intellectual activity ceases. [EMAIL PROTECTED] | -- Robert Anton Wilson http://people.debian.org/~branden/ | pgp8LQzkYx7qZ.pgp Description: PGP signature
Bug#109171: Use Maildir format by default
On Sat, Aug 18, 2001 at 08:36:05PM -0300, Cesar Eduardo Barros wrote: Debian uses the traditional mailbox style by default. However, it has some disadvantages over maildir -- one of them is that it does a non-reversible modification to the message's text if it contains the sequence 'From ' at the beginning of a line. MUAs like mutt also save the user's mail folders in the same problematic format. I'm interested in performance differences. A big flaw of the mbox format is that every byte of the file must be read to extract headers of the messages .. i.e. display a list of the messages without necessarily showing any data. Does maildir employ a summarization technique or must an MUA open every single file to extract the mailbox headers?
Bug#109171: Use Maildir format by default
On Sun, Aug 19, 2001 at 08:31:58AM -0700, Aaron Lehmann wrote: On Sat, Aug 18, 2001 at 08:36:05PM -0300, Cesar Eduardo Barros wrote: Debian uses the traditional mailbox style by default. However, it has some disadvantages over maildir -- one of them is that it does a non-reversible modification to the message's text if it contains the sequence 'From ' at the beginning of a line. MUAs like mutt also save the user's mail folders in the same problematic format. I'm interested in performance differences. A big flaw of the mbox format is that every byte of the file must be read to extract headers of the messages .. i.e. display a list of the messages without necessarily showing any data. Does maildir employ a summarization technique or must an MUA open every single file to extract the mailbox headers? See the second reference I gave in the original message... -- Cesar Eduardo Barros [EMAIL PROTECTED] [EMAIL PROTECTED]
Bug#109171: Use Maildir format by default
On Sun, Aug 19, 2001 at 02:17:40PM -0300, Cesar Eduardo Barros wrote: See the second reference I gave in the original message... That tells us that UW-imap sucks. And that's not news.
Bug#109171: Use Maildir format by default
On Aug 19, Miquel van Smoorenburg [EMAIL PROTECTED] wrote: (it usually is not, and it definitely is not for tipical POP servers). Still, deleting messages from a mbox-style mailbox means copying the entire mailbox usually at least twice. That can cause a heavy A tipical POP user downloads and deletes all messages every time. load on your mailserver. Also it is nessecary to scan the entire mbox at startup - readdir() is usually faster (if you skip the stat()) In my experience mutt is faster using mbox than maildir on my 300-500 KB inbox. -- ciao, Marco
Bug#109171: Use Maildir format by default
According to Anthony Towns: On Sun, Aug 19, 2001 at 12:51:05PM +, Miquel van Smoorenburg wrote: (it usually is not, and it definitely is not for tipical POP servers). Still, deleting messages from a mbox-style mailbox means copying the entire mailbox usually at least twice. Copying it once and renaming it, surely? Depends. If you have a quota of 10 MB on /var/spool/mail, you want the temporary copy to be somewhere else, so that means copying back forth between /tmp (or /var/spool/pop, which qpopper uses) and /var/spool/mail. Yes it's those little details that bite you in the end .. Debian should make sensible decisions about use of available technology. The reality is that maildir is only useful on some setups with remote mounted mailboxes and broken NFS locking. There certainly are mailbox formats that are better than maildir, but IMHO unix-style mbox files isn't one of them. There are? Any pointers? mbx isn't too bad - it's like mbox but with an extra index file. Cyrus imap uses a maildir style setup with one file per message but also with an index file so that it doesn't need to scan every message to get the headers or the MIME structure. Both aren't generic (though I think that mutt, exim and pine all support mbx natively) but it shows that better alternatives do exist. MIke.
Bug#109171: Use Maildir format by default
Package: debian-policy Version: 3.5.6.0 Severity: wishlist Debian uses the traditional mailbox style by default. However, it has some disadvantages over maildir -- one of them is that it does a non-reversible modification to the message's text if it contains the sequence 'From ' at the beginning of a line. MUAs like mutt also save the user's mail folders in the same problematic format. My proposal is to: 1. Use the maildir format by default for storage of the user's incoming mail. This means both the MTA must be configured for maildir by default (e.g. in eximconfig) and all the MUAs must be able to grok it without needing any manual configuration 2. Use the maildir format to store the user folders by default if the user's MUA allows it The issues are: - Debian is about choice. So, of course mailbox vs maildir will end up being a configuration question. Just make maildir the default, please. - Standards for where to put the user's incoming mail/where in $HOME to put the folders. I believe most programs which use maildir already share common places, but putting them in policy would be good. - Making programs which can't use it yet be able to use it - Somehow making programs able to guess which is the mbox/mdir choice of the day (perhaps using debconf). - Dealing with indecise people who change the default while the system is live References: 1. Content-Length Considered Harmful http://home.netscape.com/eng/mozilla/2.0/relnotes/demo/content-length.html 2. Benchmarking mbox versus maildir http://www.courier-mta.org/mbox-vs-maildir/ 3. Using maildir format http://cr.yp.to/proto/maildir.html 4. Maildir Bulletin http://www.coker.com.au/maildir-bulletin/ 5. maildir(5) http://www.qmail.org/qmail-manual-html/man5/maildir.html insert standard Debian should take the lead, look at the xterm backspace mess and how well Debian handled it, etc -- System Information Debian Release: testing/unstable Architecture: i386 Kernel: Linux flower 2.4.7 #1 Sat Jul 21 20:57:24 BRT 2001 i686 Locale: LANG=en_US.ISO8859-1, LC_CTYPE=en_US.ISO8859-1 Versions of packages debian-policy depends on: ii fileutils 4.1-2 GNU file management utilities.
