Re: [Dovecot] mbox vs. maildir storage block waste

Uh.. On 13.11.2012, at 1.02, Timo Sirainen wrote: > On 13.11.2012, at 0.44, Robin wrote: > >> On 11/11/2012 5:26 PM, Christoph Anton Mitterer wrote: >>> Have you made systematic tests? I.e. compared times for all of these >>> with those from the different dovecot backends. >> >> The choice of D

Re: [Dovecot] mbox vs. maildir storage block waste

On 13.11.2012, at 0.44, Robin wrote: > On 11/11/2012 5:26 PM, Christoph Anton Mitterer wrote: >> Have you made systematic tests? I.e. compared times for all of these >> with those from the different dovecot backends. > > The choice of Dovecot backends made no substantial difference. I used > ma

Re: [Dovecot] mbox vs. maildir storage block waste

On 11/11/2012 5:26 PM, Christoph Anton Mitterer wrote: > Have you made systematic tests? I.e. compared times for all of these > with those from the different dovecot backends. The choice of Dovecot backends made no substantial difference. I used maildir, sdbox, and mdbox. I also added SiS (with

Re: [Dovecot] mbox vs. maildir storage block waste

On Thu, 2012-11-08 at 17:54 -0800, Robin wrote: > The performance is surprisingly bad ... doing almost everything. > Searches through IMAP, bulk importation of mail folders, large > numbers of simultaneous mail deliveries, you name it. Have you made systematic tests? I.e. compared times for all o

Re: [Dovecot] mbox vs. maildir storage block waste

Christoph Anton Mitterer wrote: > On Wed, 2012-11-07 at 17:30 +0200, Timo Sirainen wrote: > > On 30.10.2012, at 2.16, Christoph Anton Mitterer wrote: > > > Have you ever thought about adding a "real" DB backend? Nothing against > > > dbox... :) ... and I have no performance comparison of dbox with

Re: [Dovecot] mbox vs. maildir storage block waste

Am 09.11.2012 02:54, schrieb Robin: > I'll stay tuned, whether we ever see a fully usable SQL backend for >> Dovecot :) thats not a new idea, but there is still tons of stuff which has to coded in more prime, as dovecot works nice with other existing storage file backends, there isnt hard pressure

Re: [Dovecot] mbox vs. maildir storage block waste

Obvious caveats and qualifications apply here throughout this email. Christoph Anton Mitterer wrote: > I see... well I haven't tested AOX or dbmail so far (especially as > they're not in Debian and I was too lazy till now to compile them)... > > At least I had the impression that performance (es

Re: [Dovecot] mbox vs. maildir storage block waste

On Wed, 2012-11-07 at 17:30 +0200, Timo Sirainen wrote: > On 30.10.2012, at 2.16, Christoph Anton Mitterer wrote: > > Have you ever thought about adding a "real" DB backend? Nothing against > > dbox... :) ... and I have no performance comparison of dbox with what > > could be done with a DBMS... bu

Re: [Dovecot] mbox vs. maildir storage block waste

On 30.10.2012, at 2.16, Christoph Anton Mitterer wrote: > Have you ever thought about adding a "real" DB backend? Nothing against > dbox... :) ... and I have no performance comparison of dbox with what > could be done with a DBMS... but the advantage of the later would be > that you get all fancy

Re: [Dovecot] mbox vs. maildir storage block waste

On 30.10.2012, at 13.00, Charles Marcus wrote: > On 2012-10-29 5:42 PM, Timo Sirainen wrote: >> On 29.10.2012, at 23.15, Christoph Anton Mitterer wrote: >> >>> btw: What are the actual advantages of sdbox over maildir? >> * Not moving files from new/ to cur/ directory >> * Not renaming files w

Re: [Dovecot] mbox vs. maildir storage block waste

On Tue, 2012-10-30 at 07:03 -0400, Charles Marcus wrote: > What makes the most sense for me is to use mbox (or mdbox) for longer > term storage that you may be offloading to slower storage systems, and > use maildir (or sdbox) for the new mails... Was also something I thought about... still the m

Re: [Dovecot] mbox vs. maildir storage block waste

On Tue, 2012-10-30 at 07:00 -0400, Charles Marcus wrote: > So... what are the disadvantages? I (but I'm no expert) would guess that it's a dovecot-only format. No support from most other tools,... I'd guess you cannot use e.g. maildrop with it, or can you? I personally was always a bit worried,

Re: [Dovecot] mbox vs. maildir storage block waste

On 2012-10-29 4:54 PM, Christoph Anton Mitterer wrote: In the end I probably changed my opinion. ~7GB of wasted block space for all my mails is actually quite a lot, but in days of cheap disk space it's acceptable. And with mbox one has IMHO the major disadvantage that mailservers (including dov

Re: [Dovecot] mbox vs. maildir storage block waste

On 2012-10-29 5:42 PM, Timo Sirainen wrote: On 29.10.2012, at 23.15, Christoph Anton Mitterer wrote: btw: What are the actual advantages of sdbox over maildir? * Not moving files from new/ to cur/ directory * Not renaming files when changing message flags * Not readdir()ing directories

Re: [Dovecot] mbox vs. maildir storage block waste

On Tue, 2012-10-30 at 00:05 +0200, Timo Sirainen wrote: > > And I guess the interior of the files is the same? I.e. just the plain > > mail without any changes or quoting? > Yes, but it's in dbox format so it contains also some extra metadata (not in > the mail headers). Yeah of course... but the

Re: [Dovecot] mbox vs. maildir storage block waste

On 29.10.2012, at 23.54, Christoph Anton Mitterer wrote: > On Mon, 2012-10-29 at 23:42 +0200, Timo Sirainen wrote: >>> btw: What are the actual advantages of sdbox over maildir? >> >> * Not moving files from new/ to cur/ directory >> * Not renaming files when changing message flags >> * Not readd

Re: [Dovecot] mbox vs. maildir storage block waste

On Mon, 2012-10-29 at 23:42 +0200, Timo Sirainen wrote: > > btw: What are the actual advantages of sdbox over maildir? > > * Not moving files from new/ to cur/ directory > * Not renaming files when changing message flags > * Not readdir()ing directories (although maildir_very_dirty_syncs=yes he

Re: [Dovecot] mbox vs. maildir storage block waste

On 29.10.2012, at 23.15, Christoph Anton Mitterer wrote: > btw: What are the actual advantages of sdbox over maildir? * Not moving files from new/ to cur/ directory * Not renaming files when changing message flags * Not readdir()ing directories (although maildir_very_dirty_syncs=yes helps a l

Re: [Dovecot] mbox vs. maildir storage block waste

On Mon, 2012-10-29 at 23:06 +0200, Timo Sirainen wrote: > There is of course mdbox also, which gives the best of both mbox and maildir > (and some of its own new annoyances). Thanks, Timo,... I forgot to mention that. For me _personally_ two things speak against using it: a) To be honest, "you m

Re: [Dovecot] mbox vs. maildir storage block waste

On 29.10.2012, at 22.54, Christoph Anton Mitterer wrote: > I recently mentioned in several posts, that I'd tended to use mbox > rather than maildir, because you don't loose so much space (due to > always allocating full blocks per maildir file and thus per mail). .. > In the end I probably changed

[Dovecot] mbox vs. maildir storage block waste

Hi. I recently mentioned in several posts, that I'd tended to use mbox rather than maildir, because you don't loose so much space (due to always allocating full blocks per maildir file and thus per mail). I made some tests of my archive, which consists of some 3,4 million mails at a total of 42G

Re: [Dovecot] mbox vs maildir

On 18/09/2010 18:43, Jim Pazarena wrote: I've had clients 'request' nested folders, and it would seem that maildir is designed with that ability while with mbox it is difficult and.or impossible to implement (nested can be achieved; but not nested AND populated in each nest level). You can, u

[Dovecot] mbox vs maildir

I've had clients 'request' nested folders, and it would seem that maildir is designed with that ability while with mbox it is difficult and.or impossible to implement (nested can be achieved; but not nested AND populated in each nest level). My question is, is one format 'better' than the other?

[Dovecot] mbox vs maildir

Ignoring my previous message just for a second, I realized that my config wasn't correct. So I have this for namespaces: namespace private { separator = / prefix = "mail/" location = mbox:~/mail:INBOX=/var/mail/%u inbox = yes hidden = no list = yes # for v1.1+ } namesp

Re: [Dovecot] mbox vs maildir

On Tue, 2007-07-10 at 13:57 -0400, FiL @ Kpoxa wrote: > >> > >> The last time I tried to convert from mbox to maildir, things got > >> pretty botched up, no data loss, but it wasn't pretty. :-) > > just because you got it wrong doesn't make it's hard. you probably > > didn't take enough time to g

Re: [Dovecot] mbox vs maildir

The last time I tried to convert from mbox to maildir, things got pretty botched up, no data loss, but it wasn't pretty. :-) just because you got it wrong doesn't make it's hard. you probably didn't take enough time to get it right. Well, if you know the RIGHT way - just share it with the r

Re: [Dovecot] mbox vs maildir

Can Dovecot handle mbox for some users and maildir for others? I'd like to try a conversion for one user... I'll probably create a new user, then have procmail copy (via ! action code) all mail for one user to that new user. I recently inherited a sendmail + UW IMAP installation (where

Re: [Dovecot] mbox vs maildir

Don Russell wrote: I'm using Dovecot 1.0.1-12 on Linux/Fedora 7 along with sendmail and procmail all running on the same box mail is stored in mbox format [snip] Thanks to all who replied. This seems to have sparked quite a discussion, and given me quite a bit to read/look into. Sounds like

Re: [Dovecot] mbox vs maildir

I did the ext3 -> ext4 switch on two of our proxyservers a few months ago. Then we forgot (!) about that test and the boxes just kept running and running and running ... Interesting... have you noticed any differences in performance? No. But at least it didn't explode in my face :) Heh.. y

Re: [Dovecot] mbox vs maildir

Ralf Hildebrandt wrote: XFS is lousy for many small files. We tried XFS for our 9000 Users (Maildir) and swithced back to ext3. Properly tuned XFS is supposedly very nice. Check out: http://everything2.com/index.pl?node_id=1479435 for more info.

Re: [Dovecot] Mbox vs maildir

On 2.7.2007, at 1.33, Simon Gao wrote: The upcoming dbox and cydir formats of course beat everything in performance :) Is dbox in Dovecot 1.1 tree now? Is it the redesigned dbox? How close is dbox ready for general use? I am interested in testing it out. Where can I find instruction on

[Dovecot] Mbox vs maildir

> The upcoming dbox and cydir formats of course beat everything in > performance :) Is dbox in Dovecot 1.1 tree now? Is it the redesigned dbox? How close is dbox ready for general use? I am interested in testing it out. Where can I find instruction on how to configure dovecot to use dbox? Si

Re: [Dovecot] mbox vs maildir

mouss wrote: >> Can Dovecot handle mbox for some users and maildir for others? I'd >> like to try a conversion for one user... I'll probably create a new >> user, then have procmail copy (via ! action code) all mail for one >> user to that new user. > > Why not use one of the available mbox 2 ma

Re: [Dovecot] mbox vs maildir

mouss wrote: mbox is broken by design. Look at the next line. From what I can tell, mbox will convert the first word of this line to ">From". This means the message is modified, which is ok for raw text, but is not ok for structure text such as TeX or XML. argh. the example doesn't even work

Re: [Dovecot] mbox vs maildir

Don Russell wrote: I'm using Dovecot 1.0.1-12 on Linux/Fedora 7 along with sendmail and procmail all running on the same box mail is stored in mbox format It's a small system with a half dozen or so e-mail "accounts". Each with 40-60MB of messages in various folders. I keep seeing messages ab

Re: [Dovecot] mbox vs maildir

* Charles Marcus <[EMAIL PROTECTED]>: > >I did the ext3 -> ext4 switch on two of our proxyservers a few months > >ago. Then we forgot (!) about that test and the boxes just kept > >running and running and running ... > > Interesting... have you noticed any differences in performance? No. But at l

Re: [Dovecot] mbox vs maildir

Curious, though - why *not* make cydir a real usable format, if its performance is so good? What if dbox's performance will be even better? We'll see. Heh... don't know why I even bothered asking - you are always about 357 steps ahead of me... ;) Is it only/because there is no good soluti

Re: [Dovecot] mbox vs maildir

On Sat, 2007-06-30 at 13:01 -0400, Charles Marcus wrote: > >> One advantage of cydir over dbox was mentioned by Mark above re > >> incremental backups - with dbox, you'd still have to backup the > >> entire mailbox file, while with cydir, you'd only have to copy > >> newer messages. > > > I was

Re: [Dovecot] mbox vs maildir

I did the ext3 -> ext4 switch on two of our proxyservers a few months ago. Then we forgot (!) about that test and the boxes just kept running and running and running ... Interesting... have you noticed any differences in performance? -- Best regards, Charles

Re: [Dovecot] mbox vs maildir

One advantage of cydir over dbox was mentioned by Mark above re incremental backups - with dbox, you'd still have to backup the entire mailbox file, while with cydir, you'd only have to copy newer messages. I was thinking about making dbox configurable. If it is run in one-mail-per-file mode

Re: [Dovecot] mbox vs maildir

* Charles Marcus <[EMAIL PROTECTED]>: > Please, lets not start that war up again! ;) > > Reiser has worked fine for me for many years, but I think the next time > I rebuild my servers I'll be using ext3, in anticipation of ext4 (since > it should be a fairly seamless switch)... I did the ext3

Re: [Dovecot] mbox vs maildir

On Fri, 2007-06-29 at 16:37 -0400, Charles Marcus wrote: > > Just remember that if you lose the index files there's no easy way to > > recover the mailbox. Well, except by copying the files to maildir.. > > > > I'm not sure if I should try to make cydir anything else than a > > benchmark format or

Re: [Dovecot] mbox vs maildir

On Fri, 2007-06-29 at 16:32 -0400, Charles Marcus wrote: > On 6/29/2007, Rick Romero ([EMAIL PROTECTED]) wrote: > > I know you can do the last with LVM on Linux, and I recall something > > similar on FreeBSD - but I have no experience with either, and > > they're both missing salvage and snapshot.

Re: [Dovecot] mbox vs maildir

On 6/29/07, Charles Marcus <[EMAIL PROTECTED]> wrote: > Just remember that if you lose the index files there's no easy way to > recover the mailbox. Well, except by copying the files to maildir.. > > I'm not sure if I should try to make cydir anything else than a > benchmark format or a simple ex

Re: [Dovecot] mbox vs maildir

Just remember that if you lose the index files there's no easy way to recover the mailbox. Well, except by copying the files to maildir.. I'm not sure if I should try to make cydir anything else than a benchmark format or a simple example for writing mail storage backends. I'm hoping that dbox wi

Re: [Dovecot] mbox vs maildir

On 6/29/2007, Rick Romero ([EMAIL PROTECTED]) wrote: I know you can do the last with LVM on Linux, and I recall something similar on FreeBSD - but I have no experience with either, and they're both missing salvage and snapshot. Eh? Guess I've just been dreaming then every time I do a snapshot o

Re: [Dovecot] mbox vs maildir

All data I have right now tells me to stay ReiserFS though. Even Dovecot's own page says XFS may not be a wise choice. My experience tells me to stay away from ReiserFS as well. Please, lets not start that war up again! ;) Reiser has worked fine for me for many years, but I think the next t

Re: [Dovecot] mbox vs maildir

On Fri, 2007-06-29 at 22:16 +0200, Nicolas KOWALSKI wrote: > As I noticed almost no difference in performance (no real numbers > here, just a usage feeling with 5-10k messages mailboxes) between > dovecot 1.0 + maildir and cyrus 2.2.13/2.3.8, having dovecot use a > storage format 10 times faster th

Re: [Dovecot] mbox vs maildir

Timo Sirainen <[EMAIL PROTECTED]> writes: > On Fri, 2007-06-29 at 21:49 +0200, Nicolas KOWALSKI wrote: >> Timo Sirainen <[EMAIL PROTECTED]> writes: >> >> > On Fri, 2007-06-29 at 20:26 +0200, Nicolas KOWALSKI wrote: >> >> Timo Sirainen <[EMAIL PROTECTED]> writes: >> >> >> >> > The upcoming dbox a

Re: [Dovecot] mbox vs maildir

On Fri, 2007-06-29 at 21:49 +0200, Nicolas KOWALSKI wrote: > Timo Sirainen <[EMAIL PROTECTED]> writes: > > > On Fri, 2007-06-29 at 20:26 +0200, Nicolas KOWALSKI wrote: > >> Timo Sirainen <[EMAIL PROTECTED]> writes: > >> > >> > The upcoming dbox and cydir formats of course beat everything in > >>

Re: [Dovecot] mbox vs maildir

Timo Sirainen <[EMAIL PROTECTED]> writes: > On Fri, 2007-06-29 at 20:26 +0200, Nicolas KOWALSKI wrote: >> Timo Sirainen <[EMAIL PROTECTED]> writes: >> >> > The upcoming dbox and cydir formats of course beat everything in >> > performance :) >> >> cydir ? Does this mean there is a cyrus-like stor

Re: [Dovecot] mbox vs maildir

On Fri, 2007-06-29 at 13:56 -0400, Jesse C. Smillie wrote: > Wow this is weird because I'm about to make this same jump next week! > > From what I'm reading so far the big draw back with mbox is the single > file with all the emails in it. When you delete a message from that > file the whole f

Re: [Dovecot] mbox vs maildir

Don Russell wrote: I'm using Dovecot 1.0.1-12 on Linux/Fedora 7 along with sendmail and procmail all running on the same box mail is stored in mbox format It's a small system with a half dozen or so e-mail "accounts". Each with 40-60MB of messages in various folders. I keep seeing messages ab

Re: [Dovecot] mbox vs maildir

On Fri, 2007-06-29 at 20:53 +0200, Ralf Hildebrandt wrote: > * Rick Romero <[EMAIL PROTECTED]>: > > > > My experience tells me to stay away from ReiserFS as well. > > > > How about NSS? I was considering using Netware's NSS for my backend > > server - either from Netware or Linux OES, but I'm n

Re: [Dovecot] mbox vs maildir

* Rick Romero <[EMAIL PROTECTED]>: > > My experience tells me to stay away from ReiserFS as well. > > How about NSS? I was considering using Netware's NSS for my backend > server - either from Netware or Linux OES, but I'm not sure how OES > actually handles the filesystem.. Anyways, in additio

Re: [Dovecot] mbox vs maildir

On Fri, 2007-06-29 at 20:34 +0200, Ralf Hildebrandt wrote: > * Jesse C. Smillie <[EMAIL PROTECTED]>: > > > The only thing I'm not sure of is what the best file system to keep this > > on. I have been keeping my home directories on ReiserFS for quite a > > while, but one of our tech thinks XFS w

Re: [Dovecot] mbox vs maildir

FWIW, I used imapsync for the data migrations of a) an old sendmail server with mbox format to a new server running postfix+dovecot, and b) and old and busted Microsoft SBS2000 Exchange instance to a new server running postfix+dovecot. Worked well in both cases, and the process left the original s

Re: [Dovecot] mbox vs maildir

On Fri, 2007-06-29 at 20:26 +0200, Nicolas KOWALSKI wrote: > Timo Sirainen <[EMAIL PROTECTED]> writes: > > > The upcoming dbox and cydir formats of course beat everything in > > performance :) > > cydir ? Does this mean there is a cyrus-like storage coming soon ? Already in v1.1 tree. It's what

Re: [Dovecot] mbox vs maildir

On Fri, 29 Jun 2007 10:14:42 -0700 Don Russell <[EMAIL PROTECTED]> wrote: > Can Dovecot handle mbox for some users and maildir for others? Yes, if you don't have the mail_location variable set, then Dovecot will look in ~/Maildir /var/mail/username ~/mail ~/Mail in that order. See http://wiki.do

Re: [Dovecot] mbox vs maildir

* Jesse C. Smillie <[EMAIL PROTECTED]>: > The only thing I'm not sure of is what the best file system to keep this > on. I have been keeping my home directories on ReiserFS for quite a > while, but one of our tech thinks XFS would be good. XFS is lousy for many small files. We tried XFS for ou

Re: [Dovecot] mbox vs maildir

Timo Sirainen <[EMAIL PROTECTED]> writes: > The upcoming dbox and cydir formats of course beat everything in > performance :) cydir ? Does this mean there is a cyrus-like storage coming soon ? -- Nicolas

Re: [Dovecot] mbox vs maildir

On Fri, 2007-06-29 at 13:56 -0400, Jesse C. Smillie wrote: > I thought this study in regards to speed was quite interesting: > http://www.courier-mta.org/mbox-vs-maildir/ > It probably doesn't have much relevance to Dovecot+mbox though. Maildir is fa

Re: [Dovecot] mbox vs maildir

Wow this is weird because I'm about to make this same jump next week! From what I'm reading so far the big draw back with mbox is the single file with all the emails in it. When you delete a message from that file the whole file has to be rewritten without that email in it. If the box is big

[Dovecot] mbox vs maildir

I'm using Dovecot 1.0.1-12 on Linux/Fedora 7 along with sendmail and procmail all running on the same box mail is stored in mbox format It's a small system with a half dozen or so e-mail "accounts". Each with 40-60MB of messages in various folders. I keep seeing messages about how mbox is anti