Funny, I've done the same in a different way. see mail2fs in contrib/nemo. Also, I have some proposal, skip to the end of the mail and let me know what you think :)
In any case, I'd love to see/try your version of upas/fs et al. Instead of adapting upas/fs, I use a mail2fs program that uses upas to convert mail into an "unpacked" form. Each mail is a directory. A "text" file contains the message text right as you would see it in a mail reader (including relative paths for attachments). Each attach is decoded and kept in the mail's directory ready to be copied, printed, etc.; if possible, using the same file name reported by the attachment. I use the very same approach for sending mail. A directory /mail/box/nemo/out is spooled to send whichever file is placed on it as a mail. The format for the outgoing messages is again similar to what you'd type in a mail writer. The whole point is that now you editor (plus couple of scripts) becomes a mail reader/writer without understanding anything about mail. My proposal: Why don't we reach an agreement and start using the very same format. I suggest keeping my mbox format but adapting upas/fs to understand it, which is a good idea. But I'm open to suggestions. thanks > From: [EMAIL PROTECTED] > To: 9fans@9fans.net > Reply-To: 9fans@9fans.net > Date: Thu Jun 12 12:10:32 CET 2008 > Subject: Re: [9fans] store 9p session-only values using lib9p > > > at the moment i'm playing only with mbox not imap or pop, i have a version > with cache per message that 'works', upas/nedmail and acme/mail are able to > read messages 'nicely', but attachments are not decoded. > > i have a couple of reservations about mbox format. first, a majority > of users that i need to support have inboxes >10mb and some > have mailboxes >100mb. thus deleting an older message will require > rewriting the whole file, and if it's not the last message, will take > up quite a bit of storage, even on venti. > > second, it's very difficult to cache. for example, suppose i have two > instances of my mail fs interpreting the same mbox at the same time. > suppose that i delete the 5th out of 500 messages with the first. the > second will then have to reread the entire mbox to get its pointers > right. even if i write an index with the first, the entire index > needs to be reread. > > third, since large messages tend to be really stinking huge, it is not > efficient to read entire messages to deduce the mime structure. (ned > reads the mime structure of each doc on startup.) that last mime part > may be tens of mb into the > > you'll notice that if one uses email in the way it was used > traditionally in unix environments, that a 500 message, inbox or 15mb > message is unreasonable. for traditional uses, mbox format and > loading the whole stinking thing into memory is a great idea. > > unfortunately, that's now how people use email these days. > our two machines running upas/fs burn up to 5gb of ram. > old rewritten mail adds several hundred mb/day to the dump. > > > also file lengths is going to be a problem if i'm going to decode files > within the fs. > > > > i've put what i have in /n/sources/contrib/gabidiaz/wip/mboxfs > > i've adapted upas/fs to use a directory for mailboxes (mdir) by default. > mailbox types that support caching (mdir and imap4) keep their cache > below Maxcache or Σ of (actively) referenced messages, whichever is bigger. > > i've also adapted the rest of upas, including deliver and marshal to > understand mdir format. imap4d should be done soon. > > this upas/fs keeps an index, so in most cases, emails are not reread unless > they are > viewed. the index contains some information -- like flags -- not recorded > in the mail. > > i'd be happy to share a working copy to those interested. but it's > not ready to be inflicted on the world yet. the "From " line date is > unresolved and imap4d still thinks it knows how to append a message > to a mailbox. > > > about the From line, qmail man page about mbox format says it is composed > of From space [EMAIL PROTECTED] space current date, and it is generated by > the delivery agent, but moving one message from a box to another doesn't use > the delivery agent :-? > > why not? this seems an arbitrary distinction between the inbox and > every other mailbox. > > - erik > >
