On Wed, Feb 3, 2010 at 12:44 PM, Hugo Monteiro <hugo.monte...@fct.unl.pt> wrote: > On 02/03/2010 10:58 AM, Scott Ryan wrote: >> >> No capability to store message blobs compressed. >> No ability to strip the meta data from the message thus being able to >> provide this info to pop3 / IMAP without using disk I/O. >> No ability to be able to link 1 attachment to multiple messages. >> No ability to move old/read messages to slower/cheaper storage >> >> All this adds up to requiring colossal amounts of high speed, >> expensive storage if you want to offer large mailboxes for millions of >> users. >> >> > > > I can see your point. > > I was just wondering if when you say "mailboxes" you do really mean > mailboxes, or maildirs?
I mean mailboxes as a 'service' to users. I essentially mean a maildir. > > Also, i should note that IMHO Qmail-LDAP should focus on the MTA part. There > are way better POP and IMAP solutions out there, like Dovecot for instance. > If someone asked me about it, i would vote for dropping the POP and IMAP > server stuff and concentrate on message acceptance and delivery. I do not totally agree - see below. > > Now about what you think should be supported: > > - Compressed message blobs. That's for reading only, right? I'm not aware of > compressed files being opened for writting on-the-fly, anywhere. If that's > the case, i state my opinion on the above paragraph. I mean the blobs being compressed and decompressed on the fly. The MTA will compress message blobs and then the POP3/IMAP4 server would decompress them reading. > > - Where should that metadata be available? Is reading messages stored in > single files (maildir), to extract metadata, that much I/O expensive than to > keep that data in a database? The meta data could be stored in a DB. This allows you to be able to break apart your storage. Use smaller amounts of fast high performance storage for you DB meta info so that your POP3/IMAP/Webmail service can provide this info quickly and use large, cheap storage for your actual message blob. Currently maildir just updates a file - you can imagine the performance hit when you are processing 1 Billion messages a month... > > - Message/File deduplication. That i would also love to see. Theoretically > it's actually rather simple to do. I just need to strip the attachments from > the messages and store them somewhere with unique names, like using the > md5sum of the file, for instance. Then your message will no longer keep the > attachment inside, but a reference to the detached attachment. You would > also need to maintain a reference on that attachment so you could know what > messages currently depend on it. The other part, of removing orphaned > attachments, would be done by the POP/IMAP server. > > - Ability to move old/read messages to slower/cheaper storage. That's not > the job of an MTA. That would be the job for an IMAP server/system. I > believe Dovecot already allows something like that. Not the job of the MTA - agreed. But having the ability to shift your meta data away from your mail, allows you to tell your POP3/IMAP4/Webmail servers where that data now lives. This is done by moving the data periodically and updating the DB accordingly. A typical example would be that you are only interested in keeping 1 week of 'new' mail on your fast, high performance storage. Old mail can be moved to slower, cheaper, larger storage > > > Anyone got more ideas/solutions? > > Regards, > > Hugo Monteiro. > > -- > fct.unl.pt:~# cat .signature > > Hugo Monteiro > Email : hugo.monte...@fct.unl.pt > Telefone : +351 212948300 Ext.15307 > Web : http://hmonteiro.net > > Divisão de Informática > Faculdade de Ciências e Tecnologia da > Universidade Nova de Lisboa > Quinta da Torre 2829-516 Caparica Portugal > Telefone: +351 212948596 Fax: +351 212948548 > www.ci.fct.unl.pt ap...@fct.unl.pt > > fct.unl.pt:~# _ > > -- Scott Ryan http://bonoboslr.wordpress.com/
