Re: Fwd: Re: Does anyone allow unlimited or extremely large quotas?
I don't actually know what sort of problems I'm referring to, hence the question. The big problem I can imagine would be opendir() and readdir() with a huge number of files in a directory, but the cyrus code doesn't appear to do that in a lot of places that would matter to a user (deleting an entire folder, delete sieve scripts, etc) in the course of normal operations. The number of files in a directory certainly seems to be the performance factor for us. We don't enforce quotas, but our largest mailboxes are only about 15Gb. Deleting large folders (~10 messages) does take some time. The only event that has troubled other users of the system was one user who had added 7.2 million messages to their trash folder, and then emptied their trash. It took the better part of a day to finish, and impacted both read and write performance for other users (nexsan providing storage over fibre, xfs on top) but the service kept going. For what it's worth the Trash folder was only a few Gb. Simon. -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Fwd: Re: Does anyone allow unlimited or extremely large quotas?
I don't actually know what sort of problems I'm referring to, hence the question. The big problem I can imagine would be opendir() and readdir() with a huge number of files in a directory, but the cyrus code doesn't appear to do that in a lot of places that would matter to a user (deleting an entire folder, delete sieve scripts, etc) in the course of normal operations. The number of files in a directory certainly seems to be the performance factor for us. We don't enforce quotas, but our largest mailboxes are only about 15Gb. Deleting large folders (~10 messages) does take some time. The only event that has troubled other users of the system was one user who had added 7.2 million messages to their trash folder, and then emptied their trash. It took the better part of a day to finish, and impacted both read and write performance for other users (nexsan providing storage over fibre, xfs on top) but the service kept Speaking of XFS it was quite slow in deleting large number of small files some years ago. If that's still true it may be what you saw. Since delayed delete has been enabled those issues have not be seen over here. Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Fwd: Re: Does anyone allow unlimited or extremely large quotas?
On Tue, 16 Nov 2010, Dave McMurtrie wrote: I didn't realize that I only responded to Rob here. Perhaps my additional information will shed some light on the kind of information I'm looking for. Original Message Subject: Re: Does anyone allow unlimited or extremely large quotas? Date: Tue, 16 Nov 2010 07:06:53 -0500 From: Dave McMurtrie dav...@andrew.cmu.edu To: Rob Mueller r...@fastmail.fm On 11/16/2010 06:45 AM, Rob Mueller wrote: This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows unlimited quota for their users or provides extremely large quotas when asked for? What do you consider extremely large? And what sort of problems are you referring to? I don't actually know what sort of problems I'm referring to, hence the question. The big problem I can imagine would be opendir() and readdir() with a huge number of files in a directory, but the cyrus code doesn't appear to do that in a lot of places that would matter to a user (deleting an entire folder, delete sieve scripts, etc) in the course of normal operations. this depends on what filesystem you are useing, I have mailboxes with hundreds of thousands of messages in them on XFS and have no problems, but on ext3 I start seeing slowdowns with a bit over ten thousand messages. The usual issue is just the huge number of emails and thus files that accumulate. Creating a fresh replica, body searching, reconstructing, etc all take quite a bit of time because of the large amount of random IOs. Apart from that, everything does actually work ok... The only issue we ever had was with a bboard that our network group sends automated system messages to. Something in their environment went haywire and we ended up with ~1.5 million messages in that bboard. They were unable to find a client that was willing to deal with the folder to be able to clean it up. I was able to connect using imtest and SELECT and FETCH messages without any problems, though. I also recall that replication was broken by this folder, but I don't remember exactly why. alpine and mulberry have no problem with huge numbers of messages. David Lang Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
