LIST is slow for 35K mailboxes

2007-10-09 Thread Ian G Batten
We have clients which issue LIST * when they start up. We think we need them to do this, because we are making quite heavy use of shared mailboxes so a mailbox may arrive in a hierarchy other than INBOX.* to which the user should subscribe. We have ~35K mailboxes (as reported by

Re: LIST is slow for 35K mailboxes

2007-10-09 Thread Blake Hudson
Joseph Brennan wrote: --On Tuesday, October 9, 2007 14:19 +0100 Ian G Batten [EMAIL PROTECTED] wrote: We have ~35K mailboxes (as reported by ctl_mboxlist -d | wc -l), and the LIST takes upwards of 5 minutes. The imapd spins as much CPU as it can get hold of, too. We have

Re: LIST is slow for 35K mailboxes

2007-10-09 Thread Wesley Craig
What's your authentication authorization infrastructure look like? Every mailbox has an associated ACL, which must be canonicalized to determine whether the user in question gets to see it. ctl_mboxlist just lists whatever it finds. I've seen Kerberos cause LIST * to take a long time

Re: LIST is slow for 35K mailboxes

2007-10-09 Thread Ian G Batten
On 09 Oct 07, at 1550, Wesley Craig wrote: What's your authentication authorization infrastructure look like? sasl into yellow pages. Every mailbox has an associated ACL, which must be canonicalized to determine whether the user in question gets to see it. ctl_mboxlist just lists

Re: LIST is slow for 35K mailboxes

2007-10-09 Thread Ian G Batten
On 09 Oct 07, at 1522, Blake Hudson wrote: Sorry this doesn't help solve your problem but it proves it should be a lot faster than that. Joseph Brennan Columbia University Information Technology Could database type differences (or contention) be an issue here? What database format are

Re: LIST is slow for 35K mailboxes

2007-10-09 Thread Ian G Batten
On 09 Oct 07, at 1522, Blake Hudson wrote: Could database type differences (or contention) be an issue here? What database format are each of you using? Yes. With skiplist (took me several stabs at it to get the conversion to work) it takes 0.19s. Versus ~250s with BDB. A slight

Re: LIST is slow for 35K mailboxes

2007-10-09 Thread Ian G Batten
On 09 Oct 07, at 1748, Ian G Batten wrote: On 09 Oct 07, at 1522, Blake Hudson wrote: Could database type differences (or contention) be an issue here? What database format are each of you using? Yes. With skiplist (took me several stabs at it to get the conversion to work) it takes

Re: LIST is slow for 35K mailboxes

2007-10-09 Thread Pascal Gienger
Ian G Batten [EMAIL PROTECTED] wrote: On 09 Oct 07, at 1522, Blake Hudson wrote: Could database type differences (or contention) be an issue here? What database format are each of you using? Yes. With skiplist (took me several stabs at it to get the conversion to work) it takes 0.19s.