Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-05-26 Thread Tim Traver
On 3/17/2010 1:21 PM, Timo Sirainen wrote: On Wed, 2010-03-17 at 13:19 -0700, Mark Moseley wrote: Since Exim wouldn't be touching the index files, is it safe to leave exim as-is and let it handle the deliveries to the maildirs natively? Exim's already got access to everything it needs

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-05-26 Thread Charles Marcus
On 2010-05-26 1:21 PM, Mark Moseley wrote: I've attached a slightly cropped rrd graph of NFS read bytes/sec on 6 mail netapps from one of our datacenters. See if you can spot where we started moving IMAP to dovecot over the course of about a month :) Wow, that's impressive... we had similar

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-05-25 Thread Mark Moseley
On Wed, Mar 17, 2010 at 3:12 PM, Mark Moseley moseleym...@gmail.com wrote: On Wed, Mar 17, 2010 at 1:09 PM, Timo Sirainen t...@iki.fi wrote: snip I don't know if anyone has run local indexes in larger setups, so I can't really give any good answers. The worst case of rebuilding the whole index

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-17 Thread Mark Moseley
First off, thanks for the reply. I appreciate it greatly! On Tue, Mar 16, 2010 at 4:36 PM, Timo Sirainen t...@iki.fi wrote: On 17.3.2010, at 1.01, Mark Moseley wrote: * Since Dovecot 2.0 seems like it's just around the corner, that's all I've been testing, and indeed all I've even looked at.

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-17 Thread Tony Rutherford
On 3/16/2010 7:36 PM, Timo Sirainen wrote: On 17.3.2010, at 1.01, Mark Moseley wrote: * Since Dovecot 2.0 seems like it's just around the corner, that's all I've been testing, and indeed all I've even looked at. Yes, hopefully it's coming soon :) * Our #1 main motivation for

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-17 Thread Timo Sirainen
On Wed, 2010-03-17 at 12:15 -0700, Mark Moseley wrote: * Our #1 main motivation for looking Dovecot is relief for our currently overtaxed NFS servers, mostly in the form of the index files. Benchmarking dovecot looks great, even with the index files in the maildir. Have you read the

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-17 Thread Mark Moseley
Oops, forgot to ask one other thing On Tue, Mar 16, 2010 at 4:36 PM, Timo Sirainen t...@iki.fi wrote: On 17.3.2010, at 1.01, Mark Moseley wrote: snip * Exim: We currently deliver all of our mail via Exim on separate servers. Our POP3/IMAP servers only do POP3/IMAP and the Exim mail

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-17 Thread Timo Sirainen
On Wed, 2010-03-17 at 13:19 -0700, Mark Moseley wrote: Since Exim wouldn't be touching the index files, is it safe to leave exim as-is and let it handle the deliveries to the maildirs natively? Exim's already got access to everything it needs including the quota. I just want to make sure I

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-17 Thread Mark Moseley
On Wed, Mar 17, 2010 at 12:43 PM, Tony Rutherford t...@bluetie.com wrote: On 3/16/2010 7:36 PM, Timo Sirainen wrote: On 17.3.2010, at 1.01, Mark Moseley wrote: * Since Dovecot 2.0 seems like it's just around the corner, that's all I've been testing, and indeed all I've even looked at.

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-17 Thread Timo Sirainen
On Wed, 2010-03-17 at 13:27 -0700, Mark Moseley wrote: I'll definitely keep that in mind. I should be able to keep things pretty segregated in terms of POP3 alone or IMAP alone but my big worry is that Courier POP3+Dovecot IMAP scenario. I know a large installation that was (is?) using

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-17 Thread Mark Moseley
On Wed, Mar 17, 2010 at 1:09 PM, Timo Sirainen t...@iki.fi wrote: On Wed, 2010-03-17 at 12:15 -0700, Mark Moseley wrote: From the wiki and from the thread, it sounds like this just affects index files. One thing I didn't see in the thread (though it'd be easy to miss in a thread that long)

[Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-16 Thread Mark Moseley
Hello to the list! I've been asked to spec out the feasibility and, if feasible, plan a migration from Courier to Dovecot for both POP3 and IMAP for about 4 million mailboxes. I've been trying to absorb all the dovecot-related info I could over the past couple of weeks from the docs and from the

Re: [Dovecot] Overly long email of miscellaneous Dovecot migration questions

2010-03-16 Thread Timo Sirainen
On 17.3.2010, at 1.01, Mark Moseley wrote: * Since Dovecot 2.0 seems like it's just around the corner, that's all I've been testing, and indeed all I've even looked at. Yes, hopefully it's coming soon :) * Our #1 main motivation for looking Dovecot is relief for our currently overtaxed NFS

Re: [Dovecot] Migration questions...

2009-05-21 Thread Richard Hobbs
pod wrote: Richard Hobbs richard.ho...@crl.toshiba.co.uk writes: 19. Once everything is working perfectly, send an email to the entire company instructing them what to do after the outage and arrange an outage and do the following steps as soon as the outage begins: a. Unplug DMZ

Re: [Dovecot] Migration questions...

2009-05-20 Thread Richard Hobbs
Hello again, people, I have been reading all of the emails sent back and forth recently, and reading many web pages, and I have a new draft of my migration plan. If anyone has the time and generosity to read through this latest plan for me and let me know if you spot any potential problems with

Re: [Dovecot] Migration questions...

2009-05-20 Thread pod
Richard Hobbs richard.ho...@crl.toshiba.co.uk writes: 19. Once everything is working perfectly, send an email to the entire company instructing them what to do after the outage and arrange an outage and do the following steps as soon as the outage begins: a. Unplug DMZ switch from

Re: [Dovecot] Migration questions...

2009-05-15 Thread Richard Hobbs
Curtis Maloney wrote: Phillip Macey wrote: Oh, we serve Maildir via Dovecot IMAP and 5000 messages per folder are a wimp. Problems start if the user: We are having some performancec issues on our server at the moment - all I can put it down to is the large size of some maildirs. Eg. `ls

Re: [Dovecot] Migration questions...

2009-05-15 Thread Richard Hobbs
Seth Mattinen wrote: Phillip Macey wrote: On 14/05/2009 5:11 PM, Steffen Kaiser wrote: On Wed, 13 May 2009, Richard Hobbs wrote: The main complaint we have from users is that their IMAP Inbox, with 5000 emails in it takes ages to appear, and no amount of coaxing will convince them to split

Re: [Dovecot] Migration questions...

2009-05-15 Thread Seth Mattinen
Richard Hobbs wrote: Seth Mattinen wrote: Phillip Macey wrote: On 14/05/2009 5:11 PM, Steffen Kaiser wrote: On Wed, 13 May 2009, Richard Hobbs wrote: The main complaint we have from users is that their IMAP Inbox, with 5000 emails in it takes ages to appear, and no amount of coaxing will

Re: [Dovecot] Migration questions...

2009-05-15 Thread Giuliano Gavazzi
On F 15 May, 2009, at 09:39 , Seth Mattinen wrote: This raises an interesting question for me actually... given that we've now decided dovecot and maildir is the way forward for us, which delivery method should we use in exim? exim can support maildir, (right?) and so can dovecot, so should

Re: [Dovecot] Migration questions...

2009-05-15 Thread Steffen Kaiser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, 15 May 2009, Giuliano Gavazzi wrote: well, using an LDA makes a little more cumbersome to check the local recipient at RCPT time. Huh? exim won't try local deliver unless it has decided it is a local recipient. You won't get overquota

Re: [Dovecot] Migration questions...

2009-05-15 Thread Richard Hobbs
Seth Mattinen wrote: Richard Hobbs wrote: Curtis Maloney wrote: Phillip Macey wrote: Oh, we serve Maildir via Dovecot IMAP and 5000 messages per folder are a wimp. Problems start if the user: We are having some performancec issues on our server at the moment - all I can put it down to is

Re: [Dovecot] Migration questions...

2009-05-15 Thread tomas
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, May 15, 2009 at 11:28:42AM +, Richard Hobbs wrote: [...] Only dovecot 'deliver' will update the index on delivery. Do does this mean that it's slightly slower to actually deliver the mail with dovecot (because it's writing two

Re: [Dovecot] Migration questions...

2009-05-15 Thread Richard Hobbs
Seth Mattinen wrote: Richard Hobbs wrote: Seth Mattinen wrote: Phillip Macey wrote: On 14/05/2009 5:11 PM, Steffen Kaiser wrote: On Wed, 13 May 2009, Richard Hobbs wrote: The main complaint we have from users is that their IMAP Inbox, with 5000 emails in it takes ages to appear, and no

Re: [Dovecot] Migration questions...

2009-05-15 Thread Jakob Hirsch
Seth Mattinen, 2009-05-15 09:39: (right?) and so can dovecot, so should i use dovecot's deliver mechanism, or exim's own internal mechanism? Only dovecot 'deliver' will update the index on delivery. That's obvious. But dovecot (AFAIK) updates (not rebuilds) indexes when it sees new

Re: [Dovecot] Migration questions...

2009-05-15 Thread Giuliano Gavazzi
On F 15 May, 2009, at 12:30 , Steffen Kaiser wrote: On Fri, 15 May 2009, Giuliano Gavazzi wrote: well, using an LDA makes a little more cumbersome to check the local recipient at RCPT time. Huh? exim won't try local deliver unless it has decided it is a local recipient. You won't get

Re: [Dovecot] Migration questions...

2009-05-14 Thread Steffen Kaiser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Wed, 13 May 2009, Richard Hobbs wrote: difference for you. A single server with recent hardware should be able to easily handle hundreds of simultaneous users. That's also good to know... i like to do a job right instead of relying on faster

Re: [Dovecot] Migration questions...

2009-05-14 Thread Seth Mattinen
Richard Hobbs wrote: That's also good to know... i like to do a job right instead of relying on faster hardware, as i'm sure you all do too, but it's good to know that if i make one or two non-optimal choices along the way, it'll probably be lightning fast anyway! The main complaint we

Re: [Dovecot] Migration questions...

2009-05-14 Thread Charles Marcus
On 5/14/2009, Steffen Kaiser (skdove...@smail.inf.fh-brs.de) wrote: b) try to move all their 20'000+ messages at once to an archive folder once a year, when their quota limit is exceeded. This can make the whole server irresponsible slow, because I have the mail_log plugin running as well.

Re: [Dovecot] Migration questions...

2009-05-14 Thread Stewart Dean
I'd point out that the big *practical* issue with mbox is the reality of big inboxes. While you can restrict the hoi polloi to something limited like a quota of under 60MB (and remember that inbox is one big honking file), the powers that be will not allow themselves to be so limited...nor

Re: [Dovecot] Migration questions...

2009-05-14 Thread Curtis Maloney
Phillip Macey wrote: Oh, we serve Maildir via Dovecot IMAP and 5000 messages per folder are a wimp. Problems start if the user: We are having some performancec issues on our server at the moment - all I can put it down to is the large size of some maildirs. Eg. `ls -ld Maildir/cur` might

Re: [Dovecot] Migration questions...

2009-05-14 Thread Seth Mattinen
Phillip Macey wrote: On 14/05/2009 5:11 PM, Steffen Kaiser wrote: On Wed, 13 May 2009, Richard Hobbs wrote: The main complaint we have from users is that their IMAP Inbox, with 5000 emails in it takes ages to appear, and no amount of coaxing will convince them to split their inbox into

Re: [Dovecot] Migration questions...

2009-05-14 Thread Timo Sirainen
On Fri, 2009-05-15 at 09:35 +1000, Phillip Macey wrote: We are having some performancec issues on our server at the moment - all I can put it down to is the large size of some maildirs. Eg. `ls -ld Maildir/cur` might show a directory 20Mb in size for some of our users (20-30k emails).

Re: [Dovecot] Migration questions...

2009-05-14 Thread Phillip Macey
On 15/05/2009 9:49 AM, Timo Sirainen wrote: On Fri, 2009-05-15 at 09:35 +1000, Phillip Macey wrote: We are having some performancec issues on our server at the moment - all I can put it down to is the large size of some maildirs. Eg. `ls -ld Maildir/cur` might show a directory 20Mb in size

Re: [Dovecot] Migration questions...

2009-05-14 Thread Timo Sirainen
On May 14, 2009, at 8:19 PM, Phillip Macey wrote: On 15/05/2009 9:49 AM, Timo Sirainen wrote: On Fri, 2009-05-15 at 09:35 +1000, Phillip Macey wrote: We are having some performancec issues on our server at the moment - all I can put it down to is the large size of some maildirs. Eg. `ls

Re: [Dovecot] Migration questions...

2009-05-14 Thread Phillip Macey
On 15/05/2009 10:33 AM, Timo Sirainen wrote: On May 14, 2009, at 8:19 PM, Phillip Macey wrote: On 15/05/2009 9:49 AM, Timo Sirainen wrote: Do you have POP3 users? What clients do your users typically use? 50% pop. 50% imap.. Roughly. With POP3 users those kinds of I/O bursts can happen

Re: [Dovecot] Migration questions...

2009-05-13 Thread Richard Hobbs
Hi All, Replies to everyone below (to keep the number of emails down)... Ed W wrote: Richard Hobbs wrote: Hi All, We are soon to migrate our mail server from one piece of hardware to another and we would like to take this opportunity to optimize things. Can I recommend you add

Re: [Dovecot] Migration questions...

2009-05-13 Thread Timo Sirainen
On May 13, 2009, at 9:57 AM, Richard Hobbs wrote: Depends on the usage, but it's significantly better performing than UW-IMAP. Dovecot+mbox is also significantly faster than UW-IMAP+mbox. OK... so Dovecot is certainly significantly faster that uw-imapd in both cases, but is dovecot fastest

Re: [Dovecot] Migration questions...

2009-05-13 Thread Richard Hobbs
Timo Sirainen wrote: On May 13, 2009, at 9:57 AM, Richard Hobbs wrote: Depends on the usage, but it's significantly better performing than UW-IMAP. Dovecot+mbox is also significantly faster than UW-IMAP+mbox. OK... so Dovecot is certainly significantly faster that uw-imapd in both cases,

Re: [Dovecot] Migration questions...

2009-05-13 Thread Scott Silva
on 5-13-2009 8:55 AM Richard Hobbs spake the following: Timo Sirainen wrote: On May 13, 2009, at 9:57 AM, Richard Hobbs wrote: Depends on the usage, but it's significantly better performing than UW-IMAP. Dovecot+mbox is also significantly faster than UW-IMAP+mbox. OK... so Dovecot is

Re: [Dovecot] Migration questions...

2009-05-13 Thread Timo Sirainen
On Wed, 2009-05-13 at 09:56 -0700, Scott Silva wrote: mbox - slow to delete mails - indexing will help this problem, but the filesystem will still have work to do in order to join the two halves of the file. Actually, I think a new file is written with everything re-written except the

Re: [Dovecot] Migration questions...

2009-05-12 Thread Richard Hobbs
Hello, Timo Sirainen wrote: On Mon, 2009-05-11 at 15:05 +, Richard Hobbs wrote: Hi All, We are soon to migrate our mail server from one piece of hardware to another and we would like to take this opportunity to optimize things. As a result, we would like to replace uw-imapd and qpopper

Re: [Dovecot] Migration questions...

2009-05-12 Thread Richard Hobbs
Sorry people - i'm replying to my own email again... my reply is below! Richard Hobbs wrote: Hello, Timo Sirainen wrote: On Mon, 2009-05-11 at 15:05 +, Richard Hobbs wrote: Hi All, We are soon to migrate our mail server from one piece of hardware to another and we would like to take

Re: [Dovecot] Migration questions...

2009-05-12 Thread Lou Duchez
I don't know whether this would help with the migration, but I routinely solve a similar problem. I have implemented mail failover between two servers -- which are configured with identical sets of mailboxes -- and every 10 minutes or so, a script grabs any E-Mails from the other server and

Re: [Dovecot] Migration questions...

2009-05-12 Thread tomas
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, May 12, 2009 at 10:09:06AM +, Richard Hobbs wrote: Hello, [...] That'd good to know. Do you happen to know where I can get a copy of this external script you speak of? Will it simply be included in the debian package (probably)? |

Re: [Dovecot] Migration questions...

2009-05-12 Thread Lou Duchez
Blast, forgot something: the Simple.pm referenced in the script is this thing: http://search.cpan.org/~jpaf/Net-IMAP-Simple-0.93/Simple.pm Download it, compile it, put it somewhere that the script can find it. I don't know whether this would help with the migration, but I routinely solve a

Re: [Dovecot] Migration questions...

2009-05-12 Thread Timo Sirainen
On May 12, 2009, at 6:41 AM, Richard Hobbs wrote: Single-dbox is the highest performing, but note that it's not as much tested as mbox and Maildir code. I think it should work ok, but I'm not aware of any larger installations using dbox currently. So in case you find a problem, you might

Re: [Dovecot] Migration questions...

2009-05-12 Thread Ed W
Richard Hobbs wrote: Hi All, We are soon to migrate our mail server from one piece of hardware to another and we would like to take this opportunity to optimize things. Can I recommend you add virtualisation to your todo list. I use linux-vserver, but there are plenty other ideas out

Re: [Dovecot] Migration questions...

2009-05-12 Thread Curtis Maloney
Richard Hobbs wrote: My colleague has mentioned something of interest... can dovecot keep the index files in RAM? If so, the performance will obviously be *so* much better than running them off the hard disks. My understanding was that in-memory indicies are discarded on logout. They're of

[Dovecot] Migration questions...

2009-05-11 Thread Richard Hobbs
Hi All, We are soon to migrate our mail server from one piece of hardware to another and we would like to take this opportunity to optimize things. As a result, we would like to replace uw-imapd and qpopper with dovecot. The version we will be installing is 1.1.13-2, as this is what is available

Re: [Dovecot] Migration questions...

2009-05-11 Thread Richard Hobbs
Hello, Apologies for replying to my own email so soon, but I've had other thoughts as well... Our server is going to have 4 hard disks. These can be configured into 2 RAID 1 (mirror) arrays, a single RAID 5 array, or a single RAID 0+1 array. Previously, we thought that two RAID 1 arrays would

Re: [Dovecot] Migration questions...

2009-05-11 Thread Timo Sirainen
On Mon, 2009-05-11 at 15:05 +, Richard Hobbs wrote: Hi All, We are soon to migrate our mail server from one piece of hardware to another and we would like to take this opportunity to optimize things. As a result, we would like to replace uw-imapd and qpopper with dovecot. The version we