On Mon, Oct 2, 2017 at 4:37 AM, Jose Marques <[email protected]> wrote: >> On 29 Sep 2017, at 02:34, Nico Kadel-Garcia <[email protected]> wrote: >> >> Storing individual messages in individual files, sacrosanct and unedited, is >> part of the basic IMAP specification. > > UW IMAP written by the author of the IMAP RFC stored multiple messages in the > same file (one file per mailbox). It also offered the option of an indexed > (binary) mailbox format for better concurrent access (less whole file > locking). The way messages are stored on the server is an implementation > detail. Having said that Exchange and Office 365 are awful in this regard.
I'm.... going to dig back into yesteryear for this stuff. Let me know if it's uninteresting, or inappropriate here, and I'll try to scale back such generalized reminescences. That's not how I remember the client: the reference client for wu-imapd was "pine", contained in the "wu-imapd" source code tarball from Washington University. That had one directory owned by the user for the entire local mailbox. That mailbox directory was defined as $HOME/ in the original implementation of the server. Folders were built as directories under the mailbox, and individual messages were individual messages were individual files. The metadata about the messages, such as whether they had been read, was encoded in the filename, so that changes of message status were file renames and extremely reliable atomic operations. This was a major performance benefit to IMAP: having to open up bulky folders and manipulate data about individual messages in them has always been a source of corruption of many database backed mail systems. The pristine message stores were invaluable for spam filter training when I did some work bringing some spam software over to 64-bit. The real performance problem was when people put too many messages in one level of one folder. The kernel operation to list many thousands of files in the same folder became increasingly tedious, but it was really easy to split one folder into multiple folders to make it manageable again. I remember having some long chats with people who never, never deleted or organized their email and splitting up their email by year into multiple folders for them, on the server side, just by grepping the "Date:" lines. The original publicly available wu-imapd software package, and the pine code contained within it, did not include SSL or support for IMAPS in its published releases. I got yelled at by Marc Crispin for stealing the wu-imapd implementation code for IMAPS and publishing it, which I *did not do*. I published pointers to places outside the US where you could get SSL patches. That mattered because sharing your IMAP passwords and content in clear text to a remote server was a quite dangerous practice, and the wu-imapd published version had no encryption. The hooks were there, the code was not. I was never sure if it was Marc, or Washington University, who elected to avoid issues with US export encryption regulations by excising the code, but it created maintenance issues if you bolted on the third party SSL tools. OpenSSL was safely available for download outside the USA, but it was not being published by commercial operating system distributors at the time, and I was working with SunOS. I also kept publishing patches so that imapd and Pine, the client included in the code base, could both be configured to use the same $HOME/mail subfolder to store a particular user's messages. Marc was absolutely convinced that a machine used for an IMAP service should use $HOME/ as the base of the IMAP directory, because *of course* you would never use a mail server as anything else or ever use an IMAP capable email client on it. And there would never be anything in your $HOME/ except your email, becuase of course you would never have working user directories on such a host. Unfortunately, some of us couldn't afford another server for just email. And as it was written, if you told imapd to use $HOME/mail, Pine would save copies of that in $HOME/mail/mail. The results were.... ugly, because using Pine on the IMAP server itself would have IMAPD report that as a folder called "mail/mail", which Pine would try to download locally as $HOME/mail/mail/mail/, and hilarity would ensue. My first college computer course was in Scheme, which had a navel gazing fascination with recursion and self-recursion. This sort of system destroying behavior is why I *loathe* careless recursion. I published those patches for imapd and pine to gracefully share $HOME/mail/ for about 10 years, and I'm very tempted to swear that Marc kept deliberately rewriting the relevant 20 lines or so of code in wu-imapd to break my patches with *every minor release*. As best I could tell, the upstream patches to that code had no other effect. The code stanzas were quite small and legible, the code was pretty good that way. When the wu-imapd codebase switched to C++, I threw in the towel, I didn't have the time to maintain my forks anymore and I no longer had to maintain my own email servers by then. > The University of St Andrews is a charity registered in Scotland, No. > SC013532.
