Re: Does anyone allow unlimited or extremely large quotas?

2010-11-23 Thread Sébastien Michel
tarball. Just sending a mail in LMTP and opening the mailbox several times (SELECT/CLOSE only) A binary diff indicated that Generation Number is incremented but nothing else. Oh yeah - expunge on close. God, that's awful. 2.4.x will fix that. Switching filehandles to read-only won't

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-23 Thread Sébastien Michel
Thanks a lot for the tip, I didn't know this tool. Sébastien 2010/11/22 Gabor Gombas gomb...@digikabel.hu On Mon, Nov 22, 2010 at 11:03:15AM +0100, Sébastien Michel wrote: Since strace doesn't help to see what mmap reads on SELECT, so I made a test on NFS server. On Linux you should

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-23 Thread Bron Gondwana
On Tue, Nov 23, 2010 at 10:09:43AM +0100, Sébastien Michel wrote: I expect that cyrus.index and cyrus.cache don't change if the client does nothing in the IMAP session, even after a SELECT. The same test in an empty mailbox has the same result, Generation Number is incremented too. Cyrus

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-23 Thread Sébastien Michel
Thanks a lot, but I know IMAP :-) I can't do anything on the client side. For mailboxes that don't change and don't have any \Deleted flag I would like to change on the server side any CLOSE by UNSELECT So upgrade to 2.4.x, it's fixed there. It was a massive amount of work, and

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-22 Thread Sébastien Michel
To be honest - it doesn't actually hurt too badly once it's in memory cache. The cyrus.cache file isn't generally needed to be entirely read, and the secret of mmap is that you only read the bits you need as you need them - it's lazily loaded. I am fully agree with you. But I don't know

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-22 Thread Bron Gondwana
On Mon, Nov 22, 2010 at 11:03:15AM +0100, Sébastien Michel wrote: To be honest - it doesn't actually hurt too badly once it's in memory cache. The cyrus.cache file isn't generally needed to be entirely read, and the secret of mmap is that you only read the bits you need as you need

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-22 Thread Pascal Gienger
Le 19 nov. 2010 à 17:48, Michel Sébastien a écrit : Is mmap still efficient ? map a gigabit file should cost a lot of I/O and a relatively long reponse time to just access the records of the most recent emails. mmap does nothing besides mapping the file as virtual memory to your process.

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-22 Thread Sébastien Michel
You have got to be kidding me. Unless there's actually something which requires the files to be rewritten (i.e. an expunge event) then this should not happen. Again, Cyrus 2.4.x will be much more efficient in this regard, only rewriting if you have explicitly enable immediate expunge

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-22 Thread Bron Gondwana
On Mon, Nov 22, 2010 at 03:36:07PM +0100, Sébastien Michel wrote: You have got to be kidding me. Unless there's actually something which requires the files to be rewritten (i.e. an expunge event) then this should not happen. Again, Cyrus 2.4.x will be much more efficient in this

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-22 Thread Gabor Gombas
On Mon, Nov 22, 2010 at 11:03:15AM +0100, Sébastien Michel wrote: Since strace doesn't help to see what mmap reads on SELECT, so I made a test on NFS server. On Linux you should use blktrace. NFS is a non-POSIX filesystem, and it may show quite different behavior compared to a POSIX filesystem

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-22 Thread Bron Gondwana
On Mon, Nov 22, 2010 at 03:36:07PM +0100, Sébastien Michel wrote: You have got to be kidding me. Unless there's actually something which requires the files to be rewritten (i.e. an expunge event) then this should not happen. Again, Cyrus 2.4.x will be much more efficient in this

RE: Does anyone allow unlimited or extremely large quotas?

2010-11-19 Thread Michel Sébastien
Our biggest currently is about 30GB I think. I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few hundred thousand messages.

RE: Does anyone allow unlimited or extremely large quotas?

2010-11-19 Thread David Carter
On Fri, 19 Nov 2010, Michel Sébastien wrote: On a 32 bit architecture: we had one folder with over a million messages which was causing processes to run out of virtual memory trying to map the cache file in. This wouldn't be a problem with a 64 bit userland. very impressive to have so much

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-19 Thread Bron Gondwana
On Fri, Nov 19, 2010 at 05:48:22PM +0100, Michel Sébastien wrote: Our biggest currently is about 30GB I think. I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-19 Thread Ciprian
Adam Tauno Williams wrote: I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few hundred thousand messages. Older versions

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-18 Thread Gavin McCullagh
Hi, On Tue, 16 Nov 2010, Adam Tauno Williams wrote: I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few hundred thousand

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-18 Thread Kenneth Marshall
On Tue, Nov 16, 2010 at 08:38:49AM -0500, Adam Tauno Williams wrote: On Tue, 2010-11-16 at 14:12 +0100, Rudy Gevaert wrote: On 11/16/2010 12:30 PM, Dave McMurtrie wrote: Good morning, This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-17 Thread Jeroen van Meeuwen (Kolab Systems)
On Tuesday, November 16, 2010 02:53:44 pm Simon Amor wrote: On 16 Nov 2010, at 13:38, Adam Tauno Williams wrote: Our largest quota's a 4GB; without any issues. I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Gavin McCullagh
Hi, On Tue, 16 Nov 2010, Dave McMurtrie wrote: This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows unlimited quota for their users or provides extremely large quotas when asked for? What do you regard as extremely large? 10GB, 100GB, 1TB, ...?

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Dave McMurtrie
On 11/16/2010 06:45 AM, Gavin McCullagh wrote: Hi, On Tue, 16 Nov 2010, Dave McMurtrie wrote: This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows unlimited quota for their users or provides extremely large quotas when asked for? What do you

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Simon Matter
On 11/16/2010 06:45 AM, Gavin McCullagh wrote: Hi, On Tue, 16 Nov 2010, Dave McMurtrie wrote: This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows unlimited quota for their users or provides extremely large quotas when asked for? What do you

Fwd: Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Dave McMurtrie
I didn't realize that I only responded to Rob here. Perhaps my additional information will shed some light on the kind of information I'm looking for. Original Message Subject: Re: Does anyone allow unlimited or extremely large quotas? Date: Tue, 16 Nov 2010 07:06:53 -0500

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Rudy Gevaert
On 11/16/2010 12:30 PM, Dave McMurtrie wrote: Good morning, This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows unlimited quota for their users or provides extremely large quotas when asked for? If so, can you describe any problems you've had with

Re: Fwd: Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Simon Fraser
I don't actually know what sort of problems I'm referring to, hence the question. The big problem I can imagine would be opendir() and readdir() with a huge number of files in a directory, but the cyrus code doesn't appear to do that in a lot of places that would matter to a user (deleting

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Adam Tauno Williams
On Tue, 2010-11-16 at 14:12 +0100, Rudy Gevaert wrote: On 11/16/2010 12:30 PM, Dave McMurtrie wrote: Good morning, This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows unlimited quota for their users or provides extremely large quotas when asked

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Mike Eggleston
On Tue, 16 Nov 2010, Adam Tauno Williams might have said: On Tue, 2010-11-16 at 14:12 +0100, Rudy Gevaert wrote: On 11/16/2010 12:30 PM, Dave McMurtrie wrote: Good morning, This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows unlimited quota

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Simon Amor
On 16 Nov 2010, at 13:38, Adam Tauno Williams wrote: Our largest quota's a 4GB; without any issues. I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit

Re: Fwd: Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Simon Matter
I don't actually know what sort of problems I'm referring to, hence the question. The big problem I can imagine would be opendir() and readdir() with a huge number of files in a directory, but the cyrus code doesn't appear to do that in a lot of places that would matter to a user (deleting

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Shuvam Misra
I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few hundred thousand messages. As far as I'm aware (the helpdesk guys

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Joseph Brennan
We know things will be fine with 10,000 messages too, but 100,000 msgs in a folder is pushing things. My inbox is 17,338 at the moment, and it's still fast. But I'm using an old copy of Mulberry, which was designed for IMAP. The problem area is clients designed for POP and adapted to IMAP.

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Kenneth Marshall
On Tue, Nov 16, 2010 at 08:41:46PM +0530, Shuvam Misra wrote: I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Wesley Craig
Didn't Dave write up.imapproxy? It makes a huge difference for, e.g., IMP roundcube. Also, configuring client to not retrieve the LIST of mailboxes during every transaction is a big win. :wes On 16 Nov 2010, at 10:11, Shuvam Misra wrote: Most programming environments in which such

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Dave McMurtrie
On 11/16/2010 10:36 AM, Wesley Craig wrote: Didn't Dave write up.imapproxy? It makes a huge difference for, e.g., IMP roundcube. Also, configuring client to not retrieve the LIST of mailboxes during every transaction is a big win. Coincidentally, yes I did originally write that :)

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Adam Tauno Williams
We find that Webmail chokes server performance much earlier than normal IMAP clients do. I know this has nothing to do with Cyrus, but I just thought I'd mention it. Most programming environments in which such Webmail thingies are written (mostly PHP on the server and nowadays lots of

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Leena Heino
On 16.11.2010 17:11, Shuvam Misra wrote: I guess Webmail is OT on a Cyrus mailing list, but can't help asking: any suggestions for improving Webmail performance? (Admission: we haven't yet tried imapproxy -- it appears to be a good piece of C which will help things.) You should install

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Wesley Craig
On 16 Nov 2010, at 10:32, Joseph Brennan wrote: I wish we'd somehow financed a native Cyrus webmail interface, that is not using IMAP but built into Cyrus. I don't think users know how good Cyrus is because they look at it through a weak intermediary. I don't think a Cyrus-specific web

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Shuvam Misra
Didn't Dave write up.imapproxy? It makes a huge difference for, e.g., IMP roundcube. Also, configuring client to not retrieve the LIST of mailboxes during every transaction is a big win. Thanks a lot -- will definitely incorporate it into our setup. How does one configure the client not to

Re: Fwd: Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread David Lang
On Tue, 16 Nov 2010, Dave McMurtrie wrote: I didn't realize that I only responded to Rob here. Perhaps my additional information will shed some light on the kind of information I'm looking for. Original Message Subject: Re: Does anyone allow unlimited or extremely large

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread David Lang
On Tue, 16 Nov 2010, Ciprian wrote: Adam Tauno Williams wrote: I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few hundred

ext3 / XFS [Was: Re: Does anyone allow unlimited or extremely large quotas?]

2010-11-16 Thread Adam Tauno Williams
On Tue, 2010-11-16 at 11:25 -0800, David Lang wrote: I don't actually know what sort of problems I'm referring to, hence the question. The big problem I can imagine would be opendir() and readdir() with a huge number of files in a directory, but the cyrus code doesn't appear to do that

Re: ext3 / XFS [Was: Re: Does anyone allow unlimited or extremely large quotas?]

2010-11-16 Thread David Lang
On Tue, 16 Nov 2010, Adam Tauno Williams wrote: On Tue, 2010-11-16 at 11:25 -0800, David Lang wrote: I don't actually know what sort of problems I'm referring to, hence the question. The big problem I can imagine would be opendir() and readdir() with a huge number of files in a directory,

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Bron Gondwana
On Tue, Nov 16, 2010 at 08:38:49AM -0500, Adam Tauno Williams wrote: On Tue, 2010-11-16 at 14:12 +0100, Rudy Gevaert wrote: On 11/16/2010 12:30 PM, Dave McMurtrie wrote: Good morning, This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows

Re: ext3 / XFS [Was: Re: Does anyone allow unlimited or extremely large quotas?]

2010-11-16 Thread Robert Mueller
This is depends on what filesystem you are useing, I have mailboxes with hundreds of thousands of messages in them on XFS and have no problems, but on ext3 I start seeing slowdowns with a bit over ten thousand messages. Was dir_index enabled on that ext3 filesystem? Prior to

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Shuvam Misra
On 16.11.2010 17:11, Shuvam Misra wrote: I guess Webmail is OT on a Cyrus mailing list, but can't help asking: any suggestions for improving Webmail performance? (Admission: we haven't yet tried imapproxy -- it appears to be a good piece of C which will help things.) You should

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Shuvam Misra
On 16 Nov 2010, at 10:32, Joseph Brennan wrote: I wish we'd somehow financed a native Cyrus webmail interface, that is not using IMAP but built into Cyrus. I don't think users know how good Cyrus is because they look at it through a weak intermediary. I don't think a Cyrus-specific web

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Pascal Gienger
Am 16.11.10 19:08, schrieb Wesley Craig: On 16 Nov 2010, at 10:32, Joseph Brennan wrote: I wish we'd somehow financed a native Cyrus webmail interface, that is not using IMAP but built into Cyrus. I don't think users know how good Cyrus is because they look at it through a weak intermediary.

Re: ext3 / XFS [Was: Re: Does anyone allow unlimited or extremely large quotas?]

2010-11-16 Thread LALOT Dominique
We use ext4 for more than one year now. Efficient and stable. A good choise 12 spool of 250GB over 10 FC disks using metalun. Dom 2010/11/16 Robert Mueller r...@fastmail.fm This is depends on what filesystem you are useing, I have mailboxes with hundreds of thousands of messages in them