Re: MESSAGE quota resource implemention

2011-09-18 Thread Greg Banks
On 17/09/11 01:37, Julien Coloos wrote: As discussed here and on IRC, I rebased my commits with all the changes: - the 'utility' methods have been promoted to 'command' ones, which are now generic and may (options hash) concern cyrus binaries - the 'start_command_bg' method is now replaced

Re: MESSAGE quota resource implemention

2011-09-16 Thread Julien Coloos
As discussed here and on IRC, I rebased my commits with all the changes: - the 'utility' methods have been promoted to 'command' ones, which are now generic and may (options hash) concern cyrus binaries - the 'start_command_bg' method is now replaced by a 'background' option in

Re: MESSAGE quota resource implemention

2011-09-15 Thread Greg Banks
On 15/09/11 03:14, Julien Coloos wrote: Le 12/09/2011 23:09, Bron Gondwana a écrit : - for 'old' mailboxes (those created before the annotation storage usage field in the index header), current annotations usage shall be computed (and added to the quota entry) upon upgrading; this way

Re: MESSAGE quota resource implemention

2011-09-14 Thread Julien Coloos
Le 12/09/2011 23:09, Bron Gondwana a écrit : - for 'old' mailboxes (those created before the annotation storage usage field in the index header), current annotations usage shall be computed (and added to the quota entry) upon upgrading; this way users won't have to run 'quota -f' for all

Re: MESSAGE quota resource implemention

2011-09-12 Thread Julien Coloos
Le 09/09/2011 14:18, Greg Banks a écrit : Ok, here you go. Not completely tested yet, so caveat emptor. Had to change two things: - mailbox_quota_check now expects a quota diff array which is good :), but change is now really applied if all diffs are ' 0' (instead of '= 0' previously); in

Re: MESSAGE quota resource implemention

2011-09-12 Thread Bron Gondwana
On Mon, Sep 12, 2011 at 04:46:52PM +0200, Julien Coloos wrote: Le 09/09/2011 14:18, Greg Banks a écrit : Ok, here you go. Not completely tested yet, so caveat emptor. Remaining things concerning annotations: - when deleting messages, annotations length is not substracted; the solution may

Re: MESSAGE quota resource implemention

2011-09-07 Thread Greg Banks
Sent from my iPhone On 08/09/2011, at 0:45, Julien Coloos julien.col...@atos.net wrote: Le 06/09/2011 10:23, Greg Banks a écrit : a) my commit Make quota -f less racy is going to cause lots of clashes (sorry!) b) Bron and I both think that your commit Compute each quota resource

Re: MESSAGE quota resource implemention

2011-09-06 Thread Greg Banks
On 05/09/11 20:16, Greg Banks wrote: On 02/09/11 20:03, Bron Gondwana wrote: On Fri, Sep 02, 2011 at 07:36:20PM +1000, Greg Banks wrote: How's about this for a strategy? When a quota resource is first enabled, (i.e. the limit is changed from UNLIMITED to some finite value), the usage is

Re: MESSAGE quota resource implemention

2011-09-06 Thread Julien Coloos
Le 06/09/2011 10:23, Greg Banks a écrit : So I think we'll need another round, sorry :( Given that the annotations quota is broken and I'll be reimplementing it anyway, you may as well ignore QUOTA_ANNOTSTORAGE in all commits, and leave out the function annotatemore_computequota() for now.

Re: MESSAGE quota resource implemention

2011-09-06 Thread Julien Coloos
Le 06/09/2011 19:48, Julien Coloos a écrit : Le 06/09/2011 10:23, Greg Banks a écrit : So I think we'll need another round, sorry :( Given that the annotations quota is broken and I'll be reimplementing it anyway, you may as well ignore QUOTA_ANNOTSTORAGE in all commits, and leave out the

Re: MESSAGE quota resource implemention

2011-09-05 Thread Bron Gondwana
On Mon, Sep 05, 2011 at 02:32:40PM +1000, Greg Banks wrote: Ok, I'm now convinced my first attempt at annotation quotas sucked too hard, here's how I want to re-implement it. Let me know what you think. - add a 32b mailbox index header entry to track the storage in bytes used by all

Re: MESSAGE quota resource implemention

2011-09-05 Thread Greg Banks
On 02/09/11 20:03, Bron Gondwana wrote: On Fri, Sep 02, 2011 at 07:36:20PM +1000, Greg Banks wrote: How's about this for a strategy? When a quota resource is first enabled, (i.e. the limit is changed from UNLIMITED to some finite value), the usage is stored as some special value which I'll

Re: MESSAGE quota resource implemention

2011-09-05 Thread Greg Banks
On 05/09/11 20:06, Bron Gondwana wrote: On Mon, Sep 05, 2011 at 02:32:40PM +1000, Greg Banks wrote: Ok, I'm now convinced my first attempt at annotation quotas sucked too hard, here's how I want to re-implement it. Let me know what you think. - add a 32b mailbox index header entry to track

Re: MESSAGE quota resource implemention

2011-09-05 Thread Julien Coloos
Le 05/09/2011 06:12, Greg Banks a écrit : Julien, I think we agreed on everything else, right? I'm looking forward to your next iteration. After picking your 'uquota_t removal' commit, I also removed it on my end, and changed the code according to our previous discussions. Adding an helper

Re: MESSAGE quota resource implemention

2011-09-05 Thread Bron Gondwana
On Mon, Sep 05, 2011 at 07:19:00PM +0200, Julien Coloos wrote: Le 05/09/2011 12:06, Bron Gondwana a écrit : On Mon, Sep 05, 2011 at 02:32:40PM +1000, Greg Banks wrote: - add a 32b mailbox index header entry to track the storage in bytes used by all annotations for the mailbox itself or for

Re: MESSAGE quota resource implemention

2011-09-02 Thread Bron Gondwana
On Fri, Sep 02, 2011 at 09:37:24AM +1000, Rob Mueller wrote: Actually, really I'd like to create a new UNIQUEID - and store all the files in paths based on uniqueid rather than on folder name. This would not only mean renames could be fast and atomic, but that delayed delete would be fast.

Re: MESSAGE quota resource implemention

2011-09-02 Thread Greg Banks
On 01/09/11 22:22, Bron Gondwana wrote: On Thu, Sep 01, 2011 at 01:27:00PM +0200, Julien Coloos wrote: Le 01/09/2011 03:03, Greg Banks a écrit : But more generally, the update the quotaroot is atomic-safe, because the mailbox doesn't add/remove things from the quotaroot racily - but quota

Re: MESSAGE quota resource implemention

2011-09-02 Thread Bron Gondwana
On Fri, Sep 02, 2011 at 07:36:20PM +1000, Greg Banks wrote: If the software was robust, underflow would not happen and we would not need to test for it and handle it. Thus the log messages are not operational messages intended for the sysadmin, but warnings about internal Cyrus problems

Re: MESSAGE quota resource implemention

2011-09-01 Thread Greg Banks
Julien Coloos wrote: Hi, As discussed on IRC last week, I worked on implementing MESSAGE quota resource in cyrus (see RFC 2087; STORAGE resource already being handled). I created a branch based on Greg's 'annotate' one on github, since his work on annotation storage management made mine a lot

Re: MESSAGE quota resource implemention

2011-09-01 Thread Julien Coloos
Hi, thanks for your comments and reviewing the code Le 01/09/2011 03:03, Greg Banks a écrit : I think that there are two things that may also be done concerning quota entries: - always recompute resource usage when changing its limit from unlimited to bounded - currently it is only done

Re: MESSAGE quota resource implemention

2011-09-01 Thread Bron Gondwana
On Thu, Sep 01, 2011 at 01:27:00PM +0200, Julien Coloos wrote: Le 01/09/2011 03:03, Greg Banks a écrit : This one's tough, I wasn't sure what to do. However, I'm happy to leave it to the administrator to have to manually run quota -f (maybe twice!) if they set a quota on a resource that is

Re: MESSAGE quota resource implemention

2011-09-01 Thread Michael Menge
Quoting Bron Gondwana br...@fastmail.fm: Actually, really I'd like to create a new UNIQUEID - and store all the files in paths based on uniqueid rather than on folder name. This would not only mean renames could be fast and atomic, but that delayed delete would be fast. The downside is a more

Re: MESSAGE quota resource implemention

2011-09-01 Thread Rob Mueller
Actually, really I'd like to create a new UNIQUEID - and store all the files in paths based on uniqueid rather than on folder name. This would not only mean renames could be fast and atomic, but that delayed delete would be fast. The downside is a more opaque filesystem layout. Oh, another

Re: MESSAGE quota resource implemention

2011-08-31 Thread Bron Gondwana
On Wed, Aug 31, 2011 at 05:50:36PM +0200, Julien Coloos wrote: Things that may be worth noting: - DUMP/UNDUMP currently does nothing special about MESSAGE or X-ANNOTATION-STORAGE quota resources - should it be transferred ? I'd like to replace DUMP/UNDUMP with replication protocol