Re: [Dovecot] mbox vs. maildir storage block waste

2012-11-12 Thread Robin
On 11/11/2012 5:26 PM, Christoph Anton Mitterer wrote: Have you made systematic tests? I.e. compared times for all of these with those from the different dovecot backends. The choice of Dovecot backends made no substantial difference. I used maildir, sdbox, and mdbox. I also added SiS (with

Re: [Dovecot] mbox vs. maildir storage block waste

2012-11-12 Thread Timo Sirainen
On 13.11.2012, at 0.44, Robin wrote: On 11/11/2012 5:26 PM, Christoph Anton Mitterer wrote: Have you made systematic tests? I.e. compared times for all of these with those from the different dovecot backends. The choice of Dovecot backends made no substantial difference. I used maildir,

Re: [Dovecot] mbox vs. maildir storage block waste

2012-11-12 Thread Timo Sirainen
Uh.. On 13.11.2012, at 1.02, Timo Sirainen wrote: On 13.11.2012, at 0.44, Robin wrote: On 11/11/2012 5:26 PM, Christoph Anton Mitterer wrote: Have you made systematic tests? I.e. compared times for all of these with those from the different dovecot backends. The choice of Dovecot

Re: [Dovecot] mbox vs. maildir storage block waste

2012-11-11 Thread Christoph Anton Mitterer
On Thu, 2012-11-08 at 17:54 -0800, Robin wrote: The performance is surprisingly bad ... doing almost everything. Searches through IMAP, bulk importation of mail folders, large numbers of simultaneous mail deliveries, you name it. Have you made systematic tests? I.e. compared times for all of

Re: [Dovecot] mbox vs. maildir storage block waste

2012-11-09 Thread Daniel Parthey
Christoph Anton Mitterer wrote: On Wed, 2012-11-07 at 17:30 +0200, Timo Sirainen wrote: On 30.10.2012, at 2.16, Christoph Anton Mitterer wrote: Have you ever thought about adding a real DB backend? Nothing against dbox... :) ... and I have no performance comparison of dbox with what

Re: [Dovecot] mbox vs. maildir storage block waste

2012-11-08 Thread Christoph Anton Mitterer
On Wed, 2012-11-07 at 17:30 +0200, Timo Sirainen wrote: On 30.10.2012, at 2.16, Christoph Anton Mitterer wrote: Have you ever thought about adding a real DB backend? Nothing against dbox... :) ... and I have no performance comparison of dbox with what could be done with a DBMS... but the

Re: [Dovecot] mbox vs. maildir storage block waste

2012-11-08 Thread Robin
Obvious caveats and qualifications apply here throughout this email. Christoph Anton Mitterer cales...@scientia.net wrote: I see... well I haven't tested AOX or dbmail so far (especially as they're not in Debian and I was too lazy till now to compile them)... At least I had the impression

Re: [Dovecot] mbox vs. maildir storage block waste

2012-11-08 Thread Robert Schetterer
Am 09.11.2012 02:54, schrieb Robin: I'll stay tuned, whether we ever see a fully usable SQL backend for Dovecot :) thats not a new idea, but there is still tons of stuff which has to coded in more prime, as dovecot works nice with other existing storage file backends, there isnt hard pressure

Re: [Dovecot] mbox vs. maildir storage block waste

2012-11-07 Thread Timo Sirainen
On 30.10.2012, at 13.00, Charles Marcus wrote: On 2012-10-29 5:42 PM, Timo Sirainen t...@iki.fi wrote: On 29.10.2012, at 23.15, Christoph Anton Mitterer wrote: btw: What are the actual advantages of sdbox over maildir? * Not moving files from new/ to cur/ directory * Not renaming files

Re: [Dovecot] mbox vs. maildir storage block waste

2012-11-07 Thread Timo Sirainen
On 30.10.2012, at 2.16, Christoph Anton Mitterer wrote: Have you ever thought about adding a real DB backend? Nothing against dbox... :) ... and I have no performance comparison of dbox with what could be done with a DBMS... but the advantage of the later would be that you get all fancy

Re: [Dovecot] mbox vs. maildir storage block waste

2012-10-30 Thread Charles Marcus
On 2012-10-29 5:42 PM, Timo Sirainen t...@iki.fi wrote: On 29.10.2012, at 23.15, Christoph Anton Mitterer wrote: btw: What are the actual advantages of sdbox over maildir? * Not moving files from new/ to cur/ directory * Not renaming files when changing message flags * Not readdir()ing

Re: [Dovecot] mbox vs. maildir storage block waste

2012-10-30 Thread Charles Marcus
On 2012-10-29 4:54 PM, Christoph Anton Mitterer cales...@scientia.net wrote: In the end I probably changed my opinion. ~7GB of wasted block space for all my mails is actually quite a lot, but in days of cheap disk space it's acceptable. And with mbox one has IMHO the major disadvantage that

Re: [Dovecot] mbox vs. maildir storage block waste

2012-10-30 Thread Christoph Anton Mitterer
On Tue, 2012-10-30 at 07:00 -0400, Charles Marcus wrote: So... what are the disadvantages? I (but I'm no expert) would guess that it's a dovecot-only format. No support from most other tools,... I'd guess you cannot use e.g. maildrop with it, or can you? I personally was always a bit worried,

Re: [Dovecot] mbox vs. maildir storage block waste

2012-10-30 Thread Christoph Anton Mitterer
On Tue, 2012-10-30 at 07:03 -0400, Charles Marcus wrote: What makes the most sense for me is to use mbox (or mdbox) for longer term storage that you may be offloading to slower storage systems, and use maildir (or sdbox) for the new mails... Was also something I thought about... still the

Re: [Dovecot] mbox vs. maildir storage block waste

2012-10-29 Thread Timo Sirainen
On 29.10.2012, at 22.54, Christoph Anton Mitterer wrote: I recently mentioned in several posts, that I'd tended to use mbox rather than maildir, because you don't loose so much space (due to always allocating full blocks per maildir file and thus per mail). .. In the end I probably changed my

Re: [Dovecot] mbox vs. maildir storage block waste

2012-10-29 Thread Christoph Anton Mitterer
On Mon, 2012-10-29 at 23:06 +0200, Timo Sirainen wrote: There is of course mdbox also, which gives the best of both mbox and maildir (and some of its own new annoyances). Thanks, Timo,... I forgot to mention that. For me _personally_ two things speak against using it: a) To be honest, you

Re: [Dovecot] mbox vs. maildir storage block waste

2012-10-29 Thread Timo Sirainen
On 29.10.2012, at 23.15, Christoph Anton Mitterer wrote: btw: What are the actual advantages of sdbox over maildir? * Not moving files from new/ to cur/ directory * Not renaming files when changing message flags * Not readdir()ing directories (although maildir_very_dirty_syncs=yes helps a

Re: [Dovecot] mbox vs. maildir storage block waste

2012-10-29 Thread Christoph Anton Mitterer
On Mon, 2012-10-29 at 23:42 +0200, Timo Sirainen wrote: btw: What are the actual advantages of sdbox over maildir? * Not moving files from new/ to cur/ directory * Not renaming files when changing message flags * Not readdir()ing directories (although maildir_very_dirty_syncs=yes helps

Re: [Dovecot] mbox vs. maildir storage block waste

2012-10-29 Thread Timo Sirainen
On 29.10.2012, at 23.54, Christoph Anton Mitterer wrote: On Mon, 2012-10-29 at 23:42 +0200, Timo Sirainen wrote: btw: What are the actual advantages of sdbox over maildir? * Not moving files from new/ to cur/ directory * Not renaming files when changing message flags * Not readdir()ing

Re: [Dovecot] mbox vs. maildir storage block waste

2012-10-29 Thread Christoph Anton Mitterer
On Tue, 2012-10-30 at 00:05 +0200, Timo Sirainen wrote: And I guess the interior of the files is the same? I.e. just the plain mail without any changes or quoting? Yes, but it's in dbox format so it contains also some extra metadata (not in the mail headers). Yeah of course... but the

Re: [Dovecot] mbox vs maildir

2007-07-10 Thread FiL @ Kpoxa
The last time I tried to convert from mbox to maildir, things got pretty botched up, no data loss, but it wasn't pretty. :-) just because you got it wrong doesn't make it's hard. you probably didn't take enough time to get it right. Well, if you know the RIGHT way - just share it with the

Re: [Dovecot] mbox vs maildir

2007-07-10 Thread Timo Sirainen
On Tue, 2007-07-10 at 13:57 -0400, FiL @ Kpoxa wrote: The last time I tried to convert from mbox to maildir, things got pretty botched up, no data loss, but it wasn't pretty. :-) just because you got it wrong doesn't make it's hard. you probably didn't take enough time to get it right.

Re: [Dovecot] mbox vs maildir

2007-07-03 Thread Tim Tsai
Can Dovecot handle mbox for some users and maildir for others? I'd like to try a conversion for one user... I'll probably create a new user, then have procmail copy (via ! action code) all mail for one user to that new user. I recently inherited a sendmail + UW IMAP installation

Re: [Dovecot] mbox vs maildir

2007-07-01 Thread Ralf Hildebrandt
* Charles Marcus [EMAIL PROTECTED]: I did the ext3 - ext4 switch on two of our proxyservers a few months ago. Then we forgot (!) about that test and the boxes just kept running and running and running ... Interesting... have you noticed any differences in performance? No. But at least it

Re: [Dovecot] mbox vs maildir

2007-07-01 Thread mouss
Don Russell wrote: I'm using Dovecot 1.0.1-12 on Linux/Fedora 7 along with sendmail and procmail all running on the same box mail is stored in mbox format It's a small system with a half dozen or so e-mail accounts. Each with 40-60MB of messages in various folders. I keep seeing messages

Re: [Dovecot] mbox vs maildir

2007-07-01 Thread mouss
mouss wrote: mbox is broken by design. Look at the next line. From what I can tell, mbox will convert the first word of this line to From. This means the message is modified, which is ok for raw text, but is not ok for structure text such as TeX or XML. argh. the example doesn't even work

Re: [Dovecot] mbox vs maildir

2007-07-01 Thread albinootje
mouss wrote: Can Dovecot handle mbox for some users and maildir for others? I'd like to try a conversion for one user... I'll probably create a new user, then have procmail copy (via ! action code) all mail for one user to that new user. Why not use one of the available mbox 2 maildir

Re: [Dovecot] mbox vs maildir

2007-07-01 Thread Michal Soltys
Ralf Hildebrandt wrote: XFS is lousy for many small files. We tried XFS for our 9000 Users (Maildir) and swithced back to ext3. Properly tuned XFS is supposedly very nice. Check out: http://everything2.com/index.pl?node_id=1479435 for more info.

Re: [Dovecot] mbox vs maildir

2007-06-30 Thread Ralf Hildebrandt
* Charles Marcus [EMAIL PROTECTED]: Please, lets not start that war up again! ;) Reiser has worked fine for me for many years, but I think the next time I rebuild my servers I'll be using ext3, in anticipation of ext4 (since it should be a fairly seamless switch)... I did the ext3 - ext4

Re: [Dovecot] mbox vs maildir

2007-06-30 Thread Timo Sirainen
On Sat, 2007-06-30 at 13:01 -0400, Charles Marcus wrote: One advantage of cydir over dbox was mentioned by Mark above re incremental backups - with dbox, you'd still have to backup the entire mailbox file, while with cydir, you'd only have to copy newer messages. I was thinking about

Re: [Dovecot] mbox vs maildir

2007-06-30 Thread Charles Marcus
Curious, though - why *not* make cydir a real usable format, if its performance is so good? What if dbox's performance will be even better? We'll see. Heh... don't know why I even bothered asking - you are always about 357 steps ahead of me... ;) Is it only/because there is no good

Re: [Dovecot] mbox vs maildir

2007-06-29 Thread Jesse C. Smillie
Wow this is weird because I'm about to make this same jump next week! From what I'm reading so far the big draw back with mbox is the single file with all the emails in it. When you delete a message from that file the whole file has to be rewritten without that email in it. If the box is

Re: [Dovecot] mbox vs maildir

2007-06-29 Thread Timo Sirainen
On Fri, 2007-06-29 at 13:56 -0400, Jesse C. Smillie wrote: I thought this study in regards to speed was quite interesting: http://www.courier-mta.org/mbox-vs-maildir/ http://www.courier-mta.org/mbox-vs-maildir/ It probably doesn't have much relevance to Dovecot+mbox though. Maildir is faster

Re: [Dovecot] mbox vs maildir

2007-06-29 Thread Nicolas KOWALSKI
Timo Sirainen [EMAIL PROTECTED] writes: The upcoming dbox and cydir formats of course beat everything in performance :) cydir ? Does this mean there is a cyrus-like storage coming soon ? -- Nicolas

Re: [Dovecot] mbox vs maildir

2007-06-29 Thread John Gateley
On Fri, 29 Jun 2007 10:14:42 -0700 Don Russell [EMAIL PROTECTED] wrote: Can Dovecot handle mbox for some users and maildir for others? Yes, if you don't have the mail_location variable set, then Dovecot will look in ~/Maildir /var/mail/username ~/mail ~/Mail in that order. See

Re: [Dovecot] mbox vs maildir

2007-06-29 Thread Rick Romero
On Fri, 2007-06-29 at 20:34 +0200, Ralf Hildebrandt wrote: * Jesse C. Smillie [EMAIL PROTECTED]: The only thing I'm not sure of is what the best file system to keep this on. I have been keeping my home directories on ReiserFS for quite a while, but one of our tech thinks XFS would be

Re: [Dovecot] mbox vs maildir

2007-06-29 Thread Mark Nienberg
Don Russell wrote: I'm using Dovecot 1.0.1-12 on Linux/Fedora 7 along with sendmail and procmail all running on the same box mail is stored in mbox format It's a small system with a half dozen or so e-mail accounts. Each with 40-60MB of messages in various folders. I keep seeing messages

Re: [Dovecot] mbox vs maildir

2007-06-29 Thread Ben Winslow
On Fri, 2007-06-29 at 13:56 -0400, Jesse C. Smillie wrote: Wow this is weird because I'm about to make this same jump next week! From what I'm reading so far the big draw back with mbox is the single file with all the emails in it. When you delete a message from that file the whole file

Re: [Dovecot] mbox vs maildir

2007-06-29 Thread Nicolas KOWALSKI
Timo Sirainen [EMAIL PROTECTED] writes: On Fri, 2007-06-29 at 20:26 +0200, Nicolas KOWALSKI wrote: Timo Sirainen [EMAIL PROTECTED] writes: The upcoming dbox and cydir formats of course beat everything in performance :) cydir ? Does this mean there is a cyrus-like storage coming soon ?

Re: [Dovecot] mbox vs maildir

2007-06-29 Thread Nicolas KOWALSKI
Timo Sirainen [EMAIL PROTECTED] writes: On Fri, 2007-06-29 at 21:49 +0200, Nicolas KOWALSKI wrote: Timo Sirainen [EMAIL PROTECTED] writes: On Fri, 2007-06-29 at 20:26 +0200, Nicolas KOWALSKI wrote: Timo Sirainen [EMAIL PROTECTED] writes: The upcoming dbox and cydir formats of course

Re: [Dovecot] mbox vs maildir

2007-06-29 Thread Timo Sirainen
On Fri, 2007-06-29 at 22:16 +0200, Nicolas KOWALSKI wrote: As I noticed almost no difference in performance (no real numbers here, just a usage feeling with 5-10k messages mailboxes) between dovecot 1.0 + maildir and cyrus 2.2.13/2.3.8, having dovecot use a storage format 10 times faster than

Re: [Dovecot] mbox vs maildir

2007-06-29 Thread Charles Marcus
All data I have right now tells me to stay ReiserFS though. Even Dovecot's own page says XFS may not be a wise choice. My experience tells me to stay away from ReiserFS as well. Please, lets not start that war up again! ;) Reiser has worked fine for me for many years, but I think the next

Re: [Dovecot] mbox vs maildir

2007-06-29 Thread Charles Marcus
On 6/29/2007, Rick Romero ([EMAIL PROTECTED]) wrote: I know you can do the last with LVM on Linux, and I recall something similar on FreeBSD - but I have no experience with either, and they're both missing salvage and snapshot. Eh? Guess I've just been dreaming then every time I do a snapshot

Re: [Dovecot] mbox vs maildir

2007-06-29 Thread Charles Marcus
Just remember that if you lose the index files there's no easy way to recover the mailbox. Well, except by copying the files to maildir.. I'm not sure if I should try to make cydir anything else than a benchmark format or a simple example for writing mail storage backends. I'm hoping that dbox

Re: [Dovecot] mbox vs maildir

2007-06-29 Thread Nicolas KOWALSKI
On 6/29/07, Charles Marcus [EMAIL PROTECTED] wrote: Just remember that if you lose the index files there's no easy way to recover the mailbox. Well, except by copying the files to maildir.. I'm not sure if I should try to make cydir anything else than a benchmark format or a simple example

Re: [Dovecot] mbox vs maildir

2007-06-29 Thread Rick Romero
On Fri, 2007-06-29 at 16:32 -0400, Charles Marcus wrote: On 6/29/2007, Rick Romero ([EMAIL PROTECTED]) wrote: I know you can do the last with LVM on Linux, and I recall something similar on FreeBSD - but I have no experience with either, and they're both missing salvage and snapshot. Eh?

Re: [Dovecot] mbox vs maildir

2007-06-29 Thread Timo Sirainen
On Fri, 2007-06-29 at 16:37 -0400, Charles Marcus wrote: Just remember that if you lose the index files there's no easy way to recover the mailbox. Well, except by copying the files to maildir.. I'm not sure if I should try to make cydir anything else than a benchmark format or a simple