Re: web mail for masses - scaling archiveopteryx - active development?

2012-11-14 Thread Mark Felder
I can't comment about scaling to this size, but it should be possible. However, I do wonder why you want to move attachments out of the database? You do realize that the attachments are also deduplicated, right? Send a 1MB attachment to a million users on your service and it will only be stored

Re: web mail for masses - scaling archiveopteryx - active development?

2012-11-14 Thread Arnt Gulbrandsen
There aren't any limitations that matter; you'll probably want to listen for a couple of database signals. mailboxes_updated is a crock and will sooner or later make trouble. I think your design makes a very great deal of sense. In fact it's about the same as what I now wish Abhijit and I had

Re: web mail for masses - scaling archiveopteryx - active development?

2012-11-14 Thread Martin Rode
On 14.11.2012 15:09, Arnt Gulbrandsen wrote: Oh. I missed that bit. Taking attachments out of the DB makes sense in two cases: - the db schema is poor (not the case for aox). - the db client library sucks. - db backups are getting too big Arnt Martin

Re: web mail for masses - scaling archiveopteryx - active development?

2012-11-14 Thread Arnt Gulbrandsen
On 11/14/2012 03:16 PM, Martin Rode wrote: - db backups are getting too big They don't grow any smaller by moving half the database from /usr/local/postgres to /usr/local/homebrew. You can shrink the backups by deleting attachments (conditionally, blah), or by not backing up the entire db.

Re: web mail for masses - scaling archiveopteryx - active development?

2012-11-14 Thread Martin Rode
On 14.11.2012 15:33, Arnt Gulbrandsen wrote: On 11/14/2012 03:16 PM, Martin Rode wrote: - db backups are getting too big They don't grow any smaller by moving half the database from /usr/local/postgres to /usr/local/homebrew. The point is, you can do incremental backups easily if you store

Re: web mail for masses - scaling archiveopteryx - active development?

2012-11-14 Thread Arnt Gulbrandsen
On 11/14/2012 03:32 PM, Martin Rode wrote: The point is, you can do incremental backups easily if you store attachments in files. If you store them inside the database a full dump gets huge and can take a lot of time. Yes. An artifact of pg_dump (and dumpall). IMO PITR backups are a better

Re: web mail for masses - scaling archiveopteryx - active development?

2012-11-14 Thread Axel Rau
Sorry, Arnt, that should go to the list: Am 14.11.2012 um 15:51 schrieb Arnt Gulbrandsen: On 11/14/2012 03:32 PM, Martin Rode wrote: The point is, you can do incremental backups easily if you store attachments in files. If you store them inside the database a full dump gets huge and can take

Re: How is 'delete' supposed to work with IMAP?

2012-11-14 Thread james
A tricky issue. How should it work when the mailbox is empty, but aox undelete would repopulate the mailbox? Arnt I think 'Empty Deleted' should mean what it says. If aox undelete can recover the data, I don't think it needs to go back to where it was, some 'salvaged' area is fine.