Re: Transfer existing quotas to new cyrus imap service
Hi James, Quoting "James B. Byrne via Info-cyrus" : We are transferring an existing cyrus-imapd 3.0.11 mailstore to another host also running cyrus-imapd 3.0.11. I cannot find any documentation on how one transfers user quotas. Can someone provide me with the link to the documentation or explain how it is done? Thanks, Cyrus stores important information in different paths. See imapd.conf manpage: configdirectory: /mailboxes.db: list of all mailboxes and acls /annotations.db: mailbox Annotations /deliver.db: Information about recent delivered mails (needed for duplicate delivery suppression and sieve vacation) /user/**: subscribed folders and seen information /quota/**: Quota-Information sievedir: Sieve Scripts Depending on the configured partitions one ore more partition-: Mailboxes/Mails of the users on the partition and optional metapartition-: Meta-Files of the Mailboxes on the partition archivepartition-: Old Mails of Mailboxes on the partition You can use cyrus-replication to transfer all user/information to the second server and keep them in sync. https://www.cyrusimap.org/imap/reference/admin/sop/replication.html Michael M.MengeTel.: (49) 7071/29-70316 Universität Tübingen Fax.: (49) 7071/29-5912 Zentrum für Datenverarbeitung mail: michael.me...@zdv.uni-tuebingen.de Wächterstraße 76 72074 Tübingen Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Transfer existing quotas to new cyrus imap service
We are transferring an existing cyrus-imapd 3.0.11 mailstore to another host also running cyrus-imapd 3.0.11. I cannot find any documentation on how one transfers user quotas. Can someone provide me with the link to the documentation or explain how it is done? Thanks, -- *** e-Mail is NOT a SECURE channel *** Do NOT transmit sensitive data via e-Mail Do NOT open attachments nor follow links sent by e-Mail James B. Byrnemailto:byrn...@harte-lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Quotas
When you say ‘delete’ do you mean a client software that may actually be moving the messages to a Trash folder? This could have the effect you mention … M -- Merlin Hartley Computer Officer MRC Mitochondrial Biology Unit University of Cambridge Cambridge, CB2 0XY United Kingdom > On 7 Nov 2018, at 13:52, James B. Byrne via Info-cyrus > wrote: > > Cyrus-IMAPD-3.0.7 on FreeBSD-11.2p4 > > We have users who, having deleted email, evidently do not recover the > quota allocated to those messages. We do not believe that we have > enabled delayed deletion. All the user mailboxes previous had been > upgraded using: > > sudo -u cyrus /usr/local/cyrus/sbin/reconstruct -f -r -G -V max user > > We have also run reconstruct. None-the-less some users show that they > have exceeded their quota even when most of their email has been > deleted. Why? > > > > > -- > *** e-Mail is NOT a SECURE channel *** >Do NOT transmit sensitive data via e-Mail > Do NOT open attachments nor follow links sent by e-Mail > > James B. Byrnemailto:byrn...@harte-lyne.ca > Harte & Lyne Limited http://www.harte-lyne.ca > 9 Brockley Drive vox: +1 905 561 1241 > Hamilton, Ontario fax: +1 905 561 0757 > Canada L8E 3C3 > > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Quotas
Le 07/11/2018 à 08:52:20-0500, James B. Byrne via Info-cyrus a écrit > Cyrus-IMAPD-3.0.7 on FreeBSD-11.2p4 > > We have users who, having deleted email, evidently do not recover the > quota allocated to those messages. We do not believe that we have Well...I don't think so. With my configuration (same as you cyrus + FreeBSD 11) when someone delete a mail the quota is recover. > enabled delayed deletion. All the user mailboxes previous had been with delayed deletion. For example : [root /usr/local/cyrus/sbin]# ./quota user.zog Quota % Used Used Resource Root 20971520 13 2889569 STORAGE user.zog 137037 MESSAGE user.zog 0 X-ANNOTATION-STORAGE user.zog 299X-NUM-FOLDERS user.zog [root /usr/local/cyrus/sbin]# cd /bals/user/zog [root /bals/user/zog]# du -s -h . 3.9G. [root /bals/user/zog]# find . -type f|wc 179261 179264 2760840 [root /bals/user/zog]# So you can see the quota show with cyrus command say I use 2889569 (~2.7 Go) and 137037 messages, but on the filesystem (ZFS) I use 3.9Go and 179000 files (event It's not precise because I didn't exclude cyrus*) Regards -- Albert SHIH DIO bâtiment 15 Observatoire de Paris xmpp: j...@obspm.fr Heure local/Local time: Wed Nov 7 16:25:13 CET 2018 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Quotas
Cyrus-IMAPD-3.0.7 on FreeBSD-11.2p4 We have users who, having deleted email, evidently do not recover the quota allocated to those messages. We do not believe that we have enabled delayed deletion. All the user mailboxes previous had been upgraded using: sudo -u cyrus /usr/local/cyrus/sbin/reconstruct -f -r -G -V max user We have also run reconstruct. None-the-less some users show that they have exceeded their quota even when most of their email has been deleted. Why? -- *** e-Mail is NOT a SECURE channel *** Do NOT transmit sensitive data via e-Mail Do NOT open attachments nor follow links sent by e-Mail James B. Byrnemailto:byrn...@harte-lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Does anyone allow unlimited or extremely large quotas?
tarball. Just sending a mail in LMTP and opening the mailbox several times (SELECT/CLOSE only) A binary diff indicated that Generation Number is incremented but nothing else. Oh yeah - expunge on close. God, that's awful. 2.4.x will fix that. Switching filehandles to read-only won't help, because the standard says to expunge on close! I expect that cyrus.index and cyrus.cache don't change if the client does nothing in the IMAP session, even after a SELECT. The same test in an empty mailbox has the same result, Generation Number is incremented too. If you actually WANT read-only, the command is called 'EXAMINE' by the way. It's like SELECT, but actually supposed to be read-only. Thanks a lot, but I know IMAP :-) I can't do anything on the client side. For mailboxes that don't change and don't have any \Deleted flag I would like to change on the server side any CLOSE by UNSELECT Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
Thanks a lot for the tip, I didn't know this tool. Sébastien 2010/11/22 Gabor Gombas gomb...@digikabel.hu On Mon, Nov 22, 2010 at 11:03:15AM +0100, Sébastien Michel wrote: Since strace doesn't help to see what mmap reads on SELECT, so I made a test on NFS server. On Linux you should use blktrace. NFS is a non-POSIX filesystem, and it may show quite different behavior compared to a POSIX filesystem on a real block device. Gabor Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On Tue, Nov 23, 2010 at 10:09:43AM +0100, Sébastien Michel wrote: I expect that cyrus.index and cyrus.cache don't change if the client does nothing in the IMAP session, even after a SELECT. The same test in an empty mailbox has the same result, Generation Number is incremented too. Cyrus 2.3.16 doesn't care - it re-packs the whole mailbox every time mailbox_expunge gets called. If you actually WANT read-only, the command is called 'EXAMINE' by the way. It's like SELECT, but actually supposed to be read-only. Thanks a lot, but I know IMAP :-) I can't do anything on the client side. For mailboxes that don't change and don't have any \Deleted flag I would like to change on the server side any CLOSE by UNSELECT So upgrade to 2.4.x, it's fixed there. It was a massive amount of work, and it's not going to backport cleanly. I've shown mailbox_expunge to our new Cyrus programmer (Greg) and he asked me to stop before we got half way through. It was over a thousand lines of complex code. Bron. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
Thanks a lot, but I know IMAP :-) I can't do anything on the client side. For mailboxes that don't change and don't have any \Deleted flag I would like to change on the server side any CLOSE by UNSELECT So upgrade to 2.4.x, it's fixed there. It was a massive amount of work, and it's not going to backport cleanly. I've shown mailbox_expunge to our new Cyrus programmer (Greg) and he asked me to stop before we got half way through. It was over a thousand lines of complex code. It seems to be a good advice. I will test such version of Cyrus. Thanks, Sébastien Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
To be honest - it doesn't actually hurt too badly once it's in memory cache. The cyrus.cache file isn't generally needed to be entirely read, and the secret of mmap is that you only read the bits you need as you need them - it's lazily loaded. I am fully agree with you. But I don't know what Cyrus really reads on SELECT to fulfill the mailbox structure. Since strace doesn't help to see what mmap reads on SELECT, so I made a test on NFS server. With a 7MB's mailbox that contains 250 emails. cyrus.index is about 20KB and cyrus.cache is about 350KB. - on SELECT, nfsstat shows 15 NFS READ = 480KB on-the-wire NFS READ. It seems that both cyrus.cache and cyrus.index are read - on CLOSE, nfsstat shows 19 NFS WRITE and strace shows that both files are rewritten With a 6GB's mailbox that contains almost 100.000 emails. cyrus.index is about 8MB and cyrus.cache is about 120MB - on SELECT nfsstat shows 300 NFS READ = 9600KB on-the-wire NFS READ. OK it is less that the size of cyrus.index and cyrus.cache - on CLOSE nfsstat shows 4105 NFS READ and 4144 NFS WRITE = 2x130MB on-the-wire NFS. In such situation mmap doesn't help because everything is read and write. I hope this behaviour can be optimized. There's no real answer if you're doing a sort on the messages, Yes I am worried about IMAP SORT and some poors IMAP clients unless you go to multiple indexes (a la database engines). That's a whole different ballgame - but the the multiplier factor gets higher. For sane sizes of N (up to 20-30 thousand messages) the O(N) of the way Cyrus does it is cheaper than a more complex database. I don't think about database but about MapReduce. Sébastien Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On Mon, Nov 22, 2010 at 11:03:15AM +0100, Sébastien Michel wrote: To be honest - it doesn't actually hurt too badly once it's in memory cache. The cyrus.cache file isn't generally needed to be entirely read, and the secret of mmap is that you only read the bits you need as you need them - it's lazily loaded. I am fully agree with you. But I don't know what Cyrus really reads on SELECT to fulfill the mailbox structure. Since strace doesn't help to see what mmap reads on SELECT, so I made a test on NFS server. Ooops. I think the NFS server might be the problem here. With a 7MB's mailbox that contains 250 emails. cyrus.index is about 20KB and cyrus.cache is about 350KB. - on SELECT, nfsstat shows 15 NFS READ = 480KB on-the-wire NFS READ. It seems that both cyrus.cache and cyrus.index are read What version of Cyrus are you testing with here? - on CLOSE, nfsstat shows 19 NFS WRITE and strace shows that both files are rewritten You have got to be kidding me. Unless there's actually something which requires the files to be rewritten (i.e. an expunge event) then this should not happen. Again, Cyrus 2.4.x will be much more efficient in this regard, only rewriting if you have explicitly enable immediate expunge rather than default expunge. With a 6GB's mailbox that contains almost 100.000 emails. cyrus.index is about 8MB and cyrus.cache is about 120MB - on SELECT nfsstat shows 300 NFS READ = 9600KB on-the-wire NFS READ. OK it is less that the size of cyrus.index and cyrus.cache - on CLOSE nfsstat shows 4105 NFS READ and 4144 NFS WRITE = 2x130MB on-the-wire NFS. In such situation mmap doesn't help because everything is read and write. I hope this behaviour can be optimized. So don't use NFS. As for read vs write. Hmm... might be more possible with Cyrus 2.4 to do this, but it's still going to be a fair bit of code complexity. I do agree with the underlying principle of don't request more access than you actually need - but the added code complexity has to be worthwhile too. Bron. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
Le 19 nov. 2010 à 17:48, Michel Sébastien a écrit : Is mmap still efficient ? map a gigabit file should cost a lot of I/O and a relatively long reponse time to just access the records of the most recent emails. mmap does nothing besides mapping the file as virtual memory to your process. Read requests on memory addresses within the mmap range yield in page-ins of pages of the mmap'd file - it behaves like swap space. if you write in this mmap'd virtual memory range then you'll trigger a pageout when the system's vm system thinks it's time to write the page out (or you unmap the file). This is a very efficient way to access a file. So mapping a Gigabyte file does not need much i/o - and 10 read accesses result in a maximum of 10 pages read. The file (or the offset given with mmap() ) must fit in the process vm, so mmap'd files can be much bigger in size on 64 bit platforms. Pascal Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
You have got to be kidding me. Unless there's actually something which requires the files to be rewritten (i.e. an expunge event) then this should not happen. Again, Cyrus 2.4.x will be much more efficient in this regard, only rewriting if you have explicitly enable immediate expunge rather than default expunge. It is a 2.3.16. Tu be sure I have tested again with a fresh downloaded tarball. Just sending a mail in LMTP and opening the mailbox several times (SELECT/CLOSE only) A binary diff indicated that Generation Number is incremented but nothing else. With a 6GB's mailbox that contains almost 100.000 emails. cyrus.index is about 8MB and cyrus.cache is about 120MB - on SELECT nfsstat shows 300 NFS READ = 9600KB on-the-wire NFS READ. OK it is less that the size of cyrus.index and cyrus.cache - on CLOSE nfsstat shows 4105 NFS READ and 4144 NFS WRITE = 2x130MB on-the-wire NFS. In such situation mmap doesn't help because everything is read and write. I hope this behaviour can be optimized. So don't use NFS. NFS is not a problem here. I have flushed the buffered cache to see what's happen. Most of the time files are cached by the kernel. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On Mon, Nov 22, 2010 at 03:36:07PM +0100, Sébastien Michel wrote: You have got to be kidding me. Unless there's actually something which requires the files to be rewritten (i.e. an expunge event) then this should not happen. Again, Cyrus 2.4.x will be much more efficient in this regard, only rewriting if you have explicitly enable immediate expunge rather than default expunge. It is a 2.3.16. Tu be sure I have tested again with a fresh downloaded tarball. Just sending a mail in LMTP and opening the mailbox several times (SELECT/CLOSE only) A binary diff indicated that Generation Number is incremented but nothing else. Oh yeah - expunge on close. God, that's awful. 2.4.x will fix that. Switching filehandles to read-only won't help, because the standard says to expunge on close! If you actually WANT read-only, the command is called 'EXAMINE' by the way. It's like SELECT, but actually supposed to be read-only. FYI - I've written an experimental open the cyrus.index and cyrus.cache files read only if the mailbox is opened with a read lock patch. It seems to work fine, but I find that the index functions actually use a write lock anyway because they could be updating \Recent data or \Seen data. Oops. So I'll have to fix that too. Bron. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On Mon, Nov 22, 2010 at 11:03:15AM +0100, Sébastien Michel wrote: Since strace doesn't help to see what mmap reads on SELECT, so I made a test on NFS server. On Linux you should use blktrace. NFS is a non-POSIX filesystem, and it may show quite different behavior compared to a POSIX filesystem on a real block device. Gabor Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On Mon, Nov 22, 2010 at 03:36:07PM +0100, Sébastien Michel wrote:
You have got to be kidding me. Unless there's actually something which
requires the files to be rewritten (i.e. an expunge event) then this
should not happen. Again, Cyrus 2.4.x will be much more efficient in
this regard, only rewriting if you have explicitly enable immediate
expunge rather than default expunge.
It is a 2.3.16. Tu be sure I have tested again with a fresh downloaded
tarball. Just sending a mail in LMTP and opening the mailbox several times
(SELECT/CLOSE only)
A binary diff indicated that Generation Number is incremented but nothing
else.
Ok - that's definitely fixed in 2.4. I'm also attaching some patches
against 2.4 if anyone else wants to have a play with them. The first
makes Cyrus open the index and cache files O_RDONLY if the index is
locked in shared mode (which implies a read-only session).
The second patch then makes index EXAMINE commands use a shared lock!
Bron.
From 2b358b943677bc50830774e465537fc040917dde Mon Sep 17 00:00:00 2001
From: Bron Gondwana br...@opera.com
Date: Tue, 23 Nov 2010 00:26:40 +1100
Subject: [PATCH 2/5] Open Files Readonly - mailbox_open_irl = read only filehandles
---
imap/mailbox.c | 26 --
imap/mailbox.h |1 +
2 files changed, 21 insertions(+), 6 deletions(-)
diff --git a/imap/mailbox.c b/imap/mailbox.c
index 1f2bbb0..8a1cafc 100644
--- a/imap/mailbox.c
+++ b/imap/mailbox.c
@@ -456,6 +456,7 @@ int mailbox_open_cache(struct mailbox *mailbox)
struct stat sbuf;
unsigned generation;
int retry = 0;
+int openflags = mailbox-is_readonly ? O_RDONLY : O_RDWR;
/* already got everything? great */
if (mailbox-cache_fd != -1 !mailbox-need_cache_refresh)
@@ -471,7 +472,7 @@ int mailbox_open_cache(struct mailbox *mailbox)
abort();
fname = mailbox_meta_fname(mailbox, META_CACHE);
- mailbox-cache_fd = open(fname, O_RDWR, 0);
+ mailbox-cache_fd = open(fname, openflags, 0);
if (mailbox-cache_fd == -1)
goto fail;
}
@@ -573,6 +574,8 @@ int mailbox_append_cache(struct mailbox *mailbox,
{
int r;
+assert(mailbox_index_islocked(mailbox, 1));
+
/* no cache content */
if (!record-crec.len)
return 0;
@@ -852,6 +855,9 @@ int mailbox_open_advanced(const char *name,
mailbox-acl = xstrdup(mbentry.acl);
mailbox-mbtype = mbentry.mbtype;
+if (index_locktype == LOCK_SHARED)
+ mailbox-is_readonly = 1;
+
r = mailbox_open_index(mailbox);
if (r) {
syslog(LOG_ERR, IOERROR: opening index %s: %m, mailbox-name);
@@ -901,6 +907,7 @@ int mailbox_open_index(struct mailbox *mailbox)
{
struct stat sbuf;
char *fname;
+int openflags = mailbox-is_readonly ? O_RDONLY : O_RDWR;
if (mailbox-i.dirty || mailbox-cache_dirty)
abort();
@@ -924,7 +931,7 @@ int mailbox_open_index(struct mailbox *mailbox)
if (!fname)
return IMAP_MAILBOX_BADNAME;
-mailbox-index_fd = open(fname, O_RDWR, 0);
+mailbox-index_fd = open(fname, openflags, 0);
if (mailbox-index_fd == -1)
return IMAP_IOERROR;
@@ -1445,12 +1452,19 @@ int mailbox_lock_index(struct mailbox *mailbox, int locktype)
restart:
-if (locktype == LOCK_EXCLUSIVE)
- r = lock_blocking(mailbox-index_fd);
-else
+if (locktype == LOCK_EXCLUSIVE) {
+ /* handle read-only case cleanly - we need to re-open read-write first! */
+ if (mailbox-is_readonly) {
+ mailbox-is_readonly = 0;
+ r = mailbox_open_index(mailbox);
+ }
+ if (!r) r = lock_blocking(mailbox-index_fd);
+}
+else {
r = lock_shared(mailbox-index_fd);
+}
-if (r == -1) {
+if (r) {
syslog(LOG_ERR, IOERROR: locking index for %s: %m,
mailbox-name);
return IMAP_IOERROR;
diff --git a/imap/mailbox.h b/imap/mailbox.h
index 77bcc1b..1f2aca3 100644
--- a/imap/mailbox.h
+++ b/imap/mailbox.h
@@ -194,6 +194,7 @@ struct mailbox {
unsigned long cache_len; /* mapped size */
int index_locktype; /* 0 = none, 1 = shared, 2 = exclusive */
+int is_readonly; /* true = open index and cache files readonly */
ino_t header_file_ino;
bit32 header_file_crc;
--
1.7.2.3
From 49e4793e97559d750571d9252074aeb388c73dab Mon Sep 17 00:00:00 2001
From: Bron Gondwana br...@opera.com
Date: Tue, 23 Nov 2010 13:34:12 +1100
Subject: [PATCH 3/5] Open with read lock in EXAMINE
---
imap/index.c | 15 +++
1 files changed, 11 insertions(+), 4 deletions(-)
diff --git a/imap/index.c b/imap/index.c
index a00dbef..77664c8 100644
--- a/imap/index.c
+++ b/imap/index.c
@@ -214,13 +214,16 @@ int index_open(const char *name, struct index_init *init,
struct index_state *state = xzmalloc(sizeof(struct index_state));
struct seqset *vanishedlist = NULL;
-r = mailbox_open_iwl(name, state-mailbox);
-if (r) goto fail;
-
if (init) {
state-myrights = cyrus_acl_myrights(init-authstate, state-mailbox-acl);
- if (init-examine_mode)
+ if (init-examine_mode) {
RE: Does anyone allow unlimited or extremely large quotas?
Our biggest currently is about 30GB I think. I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few hundred thousand messages. On a 32 bit architecture: we had one folder with over a million messages which was causing processes to run out of virtual memory trying to map the cache file in. This wouldn't be a problem with a 64 bit userland. very impressive to have so much messages in one folder therefor in one partition!. But with so many messages in one folder, I think that cyrus.index and even more cyrus.cache are huge. Is mmap still efficient ? map a gigabit file should cost a lot of I/O and a relatively long reponse time to just access the records of the most recent emails. Is it time to break the design of one cyrus.index and cyrus.cache per folder by something more scalable ? Ce message et les pièces jointes sont confidentiels et réservés à l'usage exclusif de ses destinataires. Il peut également être protégé par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir immédiatement l'expéditeur et de le détruire. L'intégrité du message ne pouvant être assurée sur Internet, la responsabilité du groupe Atos Origin ne pourra être recherchée quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'expéditeur ne donne aucune garantie à cet égard et sa responsabilité ne saurait être recherchée pour tout dommage résultant d'un virus transmis. This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos Origin group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
RE: Does anyone allow unlimited or extremely large quotas?
On Fri, 19 Nov 2010, Michel Sébastien wrote: On a 32 bit architecture: we had one folder with over a million messages which was causing processes to run out of virtual memory trying to map the cache file in. This wouldn't be a problem with a 64 bit userland. very impressive to have so much messages in one folder therefor in one partition!. We have hit this limit once, and so far only once, as well. A user was sorting their email archive (thousands of messages) generating copies to the Trash mailbox. They repeated this exercise multiple times. Each time that the user reached hit their quota limit (several GBytes), they emptied the Trash folder. Consequently the live mailbox itself never contained huge numbers of messages. However delayed expunge means that a lot of wreckage was left behind: hundreds of thousands of messages. Easily fixed by a reconstruct which discarded the obsolete information. Is mmap still efficient ? map a gigabit file should cost a lot of I/O and a relatively long reponse time to just access the records of the most recent emails. The mmap() itself has very little cost. It would only become a problem if something actually tried to read all of the cache entries, causing the data to be paged in from disk. -- David Carter Email: david.car...@ucs.cam.ac.uk University Computing Service,Phone: (01223) 334502 New Museums Site, Pembroke Street, Fax: (01223) 334679 Cambridge UK. CB2 3QH. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On Fri, Nov 19, 2010 at 05:48:22PM +0100, Michel Sébastien wrote: Our biggest currently is about 30GB I think. I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few hundred thousand messages. On a 32 bit architecture: we had one folder with over a million messages which was causing processes to run out of virtual memory trying to map the cache file in. This wouldn't be a problem with a 64 bit userland. very impressive to have so much messages in one folder therefor in one partition!. But with so many messages in one folder, I think that cyrus.index and even more cyrus.cache are huge. Is mmap still efficient ? map a gigabit file should cost a lot of I/O and a relatively long reponse time to just access the records of the most recent emails. Is it time to break the design of one cyrus.index and cyrus.cache per folder by something more scalable ? To be honest - it doesn't actually hurt too badly once it's in memory cache. The cyrus.cache file isn't generally needed to be entirely read, and the secret of mmap is that you only read the bits you need as you need them - it's lazily loaded. The cyrus.index is still pretty small - about 100MB for a million messages. It doesn't take long to speed through that. Couple of seconds at most. There's no real answer if you're doing a sort on the messages, unless you go to multiple indexes (a la database engines). That's a whole different ballgame - but the the multiplier factor gets higher. For sane sizes of N (up to 20-30 thousand messages) the O(N) of the way Cyrus does it is cheaper than a more complex database. Bron. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
Adam Tauno Williams wrote: I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few hundred thousand messages. Older versions of Outlook (e.g. Office 2003) will choke well before that (I think the limit was around 35.000 on XP SP2 ) also couple that with the local Outlook file store for that IMAP account which used to be limited to 2G. We generally advice our users to avoid going past 20.000 messages in one folder. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
Hi, On Tue, 16 Nov 2010, Adam Tauno Williams wrote: I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few hundred thousand messages. As far as I'm aware (the helpdesk guys know better than me so I'm parroting their reply), Outlook 2003's PST file has a limit of 2GB so if it's locally caching folders, you may run into that. If you use Outlook 2007 or later, the limit is more like 20GB. BUT, if you upgrade from 2003 and use the same PST, that PST may continue with the same 2GB limit. Apparently you might need to create a new PST file and move the mail into it¹. Some big users have been moved to Thunderbird to avoid this and to improve performance. Gavin ¹ To be honest, I haven't personally dealt with this issue, but this paraphrases the knowledge of those here who have. I'd think of it as having [citation required] beside it. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On Tue, Nov 16, 2010 at 08:38:49AM -0500, Adam Tauno Williams wrote: On Tue, 2010-11-16 at 14:12 +0100, Rudy Gevaert wrote: On 11/16/2010 12:30 PM, Dave McMurtrie wrote: Good morning, This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows unlimited quota for their users or provides extremely large quotas when asked for? If so, can you describe any problems you've had with this? We have users with 5 GB. Our largest quota's a 4GB; without any issues. I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few hundred thousand messages. We haven't seen any problems with them. The only general problem we face is taking backups of a full store. It just takes very long to complete full backups. (Of course that isn't tied to large or no quota.) We have quotas up to 40GB currently and depending on the use everything works very well. As others have mentioned, keeping the number of messages per folder is very important for many IMAP/POP3 clients, but if that is done everything works. Cheers, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On Tuesday, November 16, 2010 02:53:44 pm Simon Amor wrote: On 16 Nov 2010, at 13:38, Adam Tauno Williams wrote: Our largest quota's a 4GB; without any issues. I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few hundred thousand messages. Is that with the server and client on the same LAN or with the client on a low speed WAN connection? We find that 50,000 messages in a folder is more than enough to make Thunderbird/Outlook unresponsive for minutes at a time when connecting to a remote server. You may like Kontact, in these cases. Having a 36 GB online email archive myself... I find Kontact to work best with the large numbers. Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Does anyone allow unlimited or extremely large quotas?
Good morning, This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows unlimited quota for their users or provides extremely large quotas when asked for? If so, can you describe any problems you've had with this? Thanks, Dave Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
Hi, On Tue, 16 Nov 2010, Dave McMurtrie wrote: This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows unlimited quota for their users or provides extremely large quotas when asked for? What do you regard as extremely large? 10GB, 100GB, 1TB, ...? Gavin Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On 11/16/2010 06:45 AM, Gavin McCullagh wrote: Hi, On Tue, 16 Nov 2010, Dave McMurtrie wrote: This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows unlimited quota for their users or provides extremely large quotas when asked for? What do you regard as extremely large? 10GB, 100GB, 1TB, ...? Well, unlimited was the largest I had in mind. Short of that, sure, 10GB, 100GB 1TB would all be large. Thanks, Dave Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On 11/16/2010 06:45 AM, Gavin McCullagh wrote: Hi, On Tue, 16 Nov 2010, Dave McMurtrie wrote: This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows unlimited quota for their users or provides extremely large quotas when asked for? What do you regard as extremely large? 10GB, 100GB, 1TB, ...? Well, unlimited was the largest I had in mind. Short of that, sure, 10GB, 100GB 1TB would all be large. More than 10GB and unlimited is used in environments I've been busy with. Of course it really depends on the nature of service you're providing. Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Fwd: Re: Does anyone allow unlimited or extremely large quotas?
I didn't realize that I only responded to Rob here. Perhaps my additional information will shed some light on the kind of information I'm looking for. Original Message Subject: Re: Does anyone allow unlimited or extremely large quotas? Date: Tue, 16 Nov 2010 07:06:53 -0500 From: Dave McMurtrie dav...@andrew.cmu.edu To: Rob Mueller r...@fastmail.fm On 11/16/2010 06:45 AM, Rob Mueller wrote: This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows unlimited quota for their users or provides extremely large quotas when asked for? What do you consider extremely large? And what sort of problems are you referring to? I don't actually know what sort of problems I'm referring to, hence the question. The big problem I can imagine would be opendir() and readdir() with a huge number of files in a directory, but the cyrus code doesn't appear to do that in a lot of places that would matter to a user (deleting an entire folder, delete sieve scripts, etc) in the course of normal operations. My manager has asked me how well Cyrus will cope with large (100GB+) or unlimited quotas. My answer to him was that it should be okay, but I have very little practical experience with such so I wanted to ask on the list. The usual issue is just the huge number of emails and thus files that accumulate. Creating a fresh replica, body searching, reconstructing, etc all take quite a bit of time because of the large amount of random IOs. Apart from that, everything does actually work ok... The only issue we ever had was with a bboard that our network group sends automated system messages to. Something in their environment went haywire and we ended up with ~1.5 million messages in that bboard. They were unable to find a client that was willing to deal with the folder to be able to clean it up. I was able to connect using imtest and SELECT and FETCH messages without any problems, though. I also recall that replication was broken by this folder, but I don't remember exactly why. So basically, I have this tiny amount of practical experience that tells me if there are 1.5 million files in a single folder, clients may be unhappy and replication may break but the server was still generally working. Any anecdotal evidence I can collect in addition to this would be helpful for me to be able to go back to my manager with. Thanks! Dave Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On 11/16/2010 12:30 PM, Dave McMurtrie wrote: Good morning, This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows unlimited quota for their users or provides extremely large quotas when asked for? If so, can you describe any problems you've had with this? We have users with 5 GB. We haven't seen any problems with them. The only general problem we face is taking backups of a full store. It just takes very long to complete full backups. (Of course that isn't tied to large or no quota.) Rudy Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Fwd: Re: Does anyone allow unlimited or extremely large quotas?
I don't actually know what sort of problems I'm referring to, hence the question. The big problem I can imagine would be opendir() and readdir() with a huge number of files in a directory, but the cyrus code doesn't appear to do that in a lot of places that would matter to a user (deleting an entire folder, delete sieve scripts, etc) in the course of normal operations. The number of files in a directory certainly seems to be the performance factor for us. We don't enforce quotas, but our largest mailboxes are only about 15Gb. Deleting large folders (~10 messages) does take some time. The only event that has troubled other users of the system was one user who had added 7.2 million messages to their trash folder, and then emptied their trash. It took the better part of a day to finish, and impacted both read and write performance for other users (nexsan providing storage over fibre, xfs on top) but the service kept going. For what it's worth the Trash folder was only a few Gb. Simon. -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On Tue, 2010-11-16 at 14:12 +0100, Rudy Gevaert wrote: On 11/16/2010 12:30 PM, Dave McMurtrie wrote: Good morning, This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows unlimited quota for their users or provides extremely large quotas when asked for? If so, can you describe any problems you've had with this? We have users with 5 GB. Our largest quota's a 4GB; without any issues. I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few hundred thousand messages. We haven't seen any problems with them. The only general problem we face is taking backups of a full store. It just takes very long to complete full backups. (Of course that isn't tied to large or no quota.) Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On Tue, 16 Nov 2010, Adam Tauno Williams might have said: On Tue, 2010-11-16 at 14:12 +0100, Rudy Gevaert wrote: On 11/16/2010 12:30 PM, Dave McMurtrie wrote: Good morning, This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows unlimited quota for their users or provides extremely large quotas when asked for? If so, can you describe any problems you've had with this? We have users with 5 GB. Our largest quota's a 4GB; without any issues. I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few hundred thousand messages. We haven't seen any problems with them. The only general problem we face is taking backups of a full store. It just takes very long to complete full backups. (Of course that isn't tied to large or no quota.) I have two users that have over 4GB of email. I manage a small shop, around 25 users depending on the time of year. The support and sales people insist on keeping all mail messages ad infitium. I have no quotas. Mike Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On 16 Nov 2010, at 13:38, Adam Tauno Williams wrote: Our largest quota's a 4GB; without any issues. I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few hundred thousand messages. Is that with the server and client on the same LAN or with the client on a low speed WAN connection? We find that 50,000 messages in a folder is more than enough to make Thunderbird/Outlook unresponsive for minutes at a time when connecting to a remote server. Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Fwd: Re: Does anyone allow unlimited or extremely large quotas?
I don't actually know what sort of problems I'm referring to, hence the question. The big problem I can imagine would be opendir() and readdir() with a huge number of files in a directory, but the cyrus code doesn't appear to do that in a lot of places that would matter to a user (deleting an entire folder, delete sieve scripts, etc) in the course of normal operations. The number of files in a directory certainly seems to be the performance factor for us. We don't enforce quotas, but our largest mailboxes are only about 15Gb. Deleting large folders (~10 messages) does take some time. The only event that has troubled other users of the system was one user who had added 7.2 million messages to their trash folder, and then emptied their trash. It took the better part of a day to finish, and impacted both read and write performance for other users (nexsan providing storage over fibre, xfs on top) but the service kept Speaking of XFS it was quite slow in deleting large number of small files some years ago. If that's still true it may be what you saw. Since delayed delete has been enabled those issues have not be seen over here. Simon Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few hundred thousand messages. As far as I'm aware (the helpdesk guys know better than me so I'm parroting their reply), Outlook 2003's PST file has a limit of 2GB so if it's locally caching folders, you may run into that. If you use Outlook 2007 or later, the limit is more like 20GB. BUT, if you upgrade from 2003 and use the same PST, that PST may continue with the same 2GB limit. Apparently you might need to create a new PST file and move the mail into it?. Some big users have been moved to Thunderbird to avoid this and to improve performance. Gavin ? To be honest, I haven't personally dealt with this issue, but this paraphrases the knowledge of those here who have. I'd think of it as having [citation required] beside it. I'm in an almost-identical position w.r.t. lack of direct knowledge, but our Support guys say exactly the same things about Outlook 2003 and Outlook 2007 and size limits of PST files. That said, we have users of our product who have 40GB mailboxes. Cyrus works perfectly happily with all this. The problem is the number of messages in the current folder, as many have mentioned before me. We keep telling users to clean up their Inboxes and keep a max of 1,000 msgs there. We know things will be fine with 10,000 messages too, but 100,000 msgs in a folder is pushing things. We find that Webmail chokes server performance much earlier than normal IMAP clients do. I know this has nothing to do with Cyrus, but I just thought I'd mention it. Most programming environments in which such Webmail thingies are written (mostly PHP on the server and nowadays lots of Javascript on the browser) cannot keep an IMAP connection to the Cyrus server open between pages, therefore each time a user clicks on a folder or does any other operation, there's this fresh IMAP connection and a huge surge of IMAP operations while the folder contents are listed afresh, etc. This puts a lot of load on the server. I guess Webmail is OT on a Cyrus mailing list, but can't help asking: any suggestions for improving Webmail performance? (Admission: we haven't yet tried imapproxy -- it appears to be a good piece of C which will help things.) regards, Shuvam Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
We know things will be fine with 10,000 messages too, but 100,000 msgs in a folder is pushing things. My inbox is 17,338 at the moment, and it's still fast. But I'm using an old copy of Mulberry, which was designed for IMAP. The problem area is clients designed for POP and adapted to IMAP. I'd say this relevant to Cyrus, because if we have a great server side but weak user-facing software, we lose the game. I wish we'd somehow financed a native Cyrus webmail interface, that is not using IMAP but built into Cyrus. I don't think users know how good Cyrus is because they look at it through a weak intermediary. (Admission: we haven't yet tried imapproxy -- it appears to be a good piece of C which will help things.) Do it. It makes a huge difference. You go from crawling to just slow. Joseph Brennan Columbia University Information Technology Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On Tue, Nov 16, 2010 at 08:41:46PM +0530, Shuvam Misra wrote: I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few hundred thousand messages. As far as I'm aware (the helpdesk guys know better than me so I'm parroting their reply), Outlook 2003's PST file has a limit of 2GB so if it's locally caching folders, you may run into that. If you use Outlook 2007 or later, the limit is more like 20GB. BUT, if you upgrade from 2003 and use the same PST, that PST may continue with the same 2GB limit. Apparently you might need to create a new PST file and move the mail into it?. Some big users have been moved to Thunderbird to avoid this and to improve performance. Gavin ? To be honest, I haven't personally dealt with this issue, but this paraphrases the knowledge of those here who have. I'd think of it as having [citation required] beside it. I'm in an almost-identical position w.r.t. lack of direct knowledge, but our Support guys say exactly the same things about Outlook 2003 and Outlook 2007 and size limits of PST files. That said, we have users of our product who have 40GB mailboxes. Cyrus works perfectly happily with all this. The problem is the number of messages in the current folder, as many have mentioned before me. We keep telling users to clean up their Inboxes and keep a max of 1,000 msgs there. We know things will be fine with 10,000 messages too, but 100,000 msgs in a folder is pushing things. We find that Webmail chokes server performance much earlier than normal IMAP clients do. I know this has nothing to do with Cyrus, but I just thought I'd mention it. Most programming environments in which such Webmail thingies are written (mostly PHP on the server and nowadays lots of Javascript on the browser) cannot keep an IMAP connection to the Cyrus server open between pages, therefore each time a user clicks on a folder or does any other operation, there's this fresh IMAP connection and a huge surge of IMAP operations while the folder contents are listed afresh, etc. This puts a lot of load on the server. I guess Webmail is OT on a Cyrus mailing list, but can't help asking: any suggestions for improving Webmail performance? (Admission: we haven't yet tried imapproxy -- it appears to be a good piece of C which will help things.) regards, Shuvam We use imapproxy here to avoid exactly this situation with webmail. Cheers, Ken Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
Didn't Dave write up.imapproxy? It makes a huge difference for, e.g., IMP roundcube. Also, configuring client to not retrieve the LIST of mailboxes during every transaction is a big win. :wes On 16 Nov 2010, at 10:11, Shuvam Misra wrote: Most programming environments in which such Webmail thingies are written (mostly PHP on the server and nowadays lots of Javascript on the browser) cannot keep an IMAP connection to the Cyrus server open between pages, therefore each time a user clicks on a folder or does any other operation, there's this fresh IMAP connection and a huge surge of IMAP operations while the folder contents are listed afresh, etc. This puts a lot of load on the server. I guess Webmail is OT on a Cyrus mailing list, but can't help asking: any suggestions for improving Webmail performance? Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On 11/16/2010 10:36 AM, Wesley Craig wrote: Didn't Dave write up.imapproxy? It makes a huge difference for, e.g., IMP roundcube. Also, configuring client to not retrieve the LIST of mailboxes during every transaction is a big win. Coincidentally, yes I did originally write that :) Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
We find that Webmail chokes server performance much earlier than normal IMAP clients do. I know this has nothing to do with Cyrus, but I just thought I'd mention it. Most programming environments in which such Webmail thingies are written (mostly PHP on the server and nowadays lots of Javascript on the browser) cannot keep an IMAP connection to the Cyrus server open between pages, therefore each time a user clicks on a folder or does any other operation, there's this fresh IMAP connection and a huge surge of IMAP operations while the folder contents are listed afresh, etc. This puts a lot of load on the server. I guess Webmail is OT on a Cyrus mailing list, but can't help asking: any suggestions for improving Webmail performance? (Admission: we haven't yet tried imapproxy -- it appears to be a good piece of C which will help things.) We use imapproxy here to avoid exactly this situation with webmail. +1 We use up.imapproxy; since installing that our Horde/IMP servers *fly*. The performance difference is really impressive. -- Adam Tauno Williams awill...@whitemice.org LPIC-1, Novell CLA http://www.whitemiceconsulting.com OpenGroupware, Cyrus IMAPd, Postfix, OpenLDAP, Samba Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On 16.11.2010 17:11, Shuvam Misra wrote: I guess Webmail is OT on a Cyrus mailing list, but can't help asking: any suggestions for improving Webmail performance? (Admission: we haven't yet tried imapproxy -- it appears to be a good piece of C which will help things.) You should install imapproxy. Also make sure you do a lot of caching on the webmail side for example if the webmail is programmed with php use apc and cache mailbox listing and message headers if possible. Cache also things on the imap server, if your cyrus version supports it you should try to use status cache and test whether it makes any difference in your environment. -- Leena Heino University of Tampere / Computer Centre ( liinu at uta.fi ) ( http://www.uta.fi/laitokset/tkk ) Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On 16 Nov 2010, at 10:32, Joseph Brennan wrote: I wish we'd somehow financed a native Cyrus webmail interface, that is not using IMAP but built into Cyrus. I don't think users know how good Cyrus is because they look at it through a weak intermediary. I don't think a Cyrus-specific web interface is the answer to that question. IMP performance is not great, but it's the http paradigm that slows it. Check out roundcube, utilizing AJAX it's way more responsive to the user. :wes (UMich runs IMP roundcube both pointing to unlimited quota Cyrus servers. Roundcube is the win.) Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
Didn't Dave write up.imapproxy? It makes a huge difference for, e.g., IMP roundcube. Also, configuring client to not retrieve the LIST of mailboxes during every transaction is a big win. Thanks a lot -- will definitely incorporate it into our setup. How does one configure the client not to retrieve the LIST of mailboxes? Can these be set for, say, roundcube or SqMail? Shuvam Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Fwd: Re: Does anyone allow unlimited or extremely large quotas?
On Tue, 16 Nov 2010, Dave McMurtrie wrote: I didn't realize that I only responded to Rob here. Perhaps my additional information will shed some light on the kind of information I'm looking for. Original Message Subject: Re: Does anyone allow unlimited or extremely large quotas? Date: Tue, 16 Nov 2010 07:06:53 -0500 From: Dave McMurtrie dav...@andrew.cmu.edu To: Rob Mueller r...@fastmail.fm On 11/16/2010 06:45 AM, Rob Mueller wrote: This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows unlimited quota for their users or provides extremely large quotas when asked for? What do you consider extremely large? And what sort of problems are you referring to? I don't actually know what sort of problems I'm referring to, hence the question. The big problem I can imagine would be opendir() and readdir() with a huge number of files in a directory, but the cyrus code doesn't appear to do that in a lot of places that would matter to a user (deleting an entire folder, delete sieve scripts, etc) in the course of normal operations. this depends on what filesystem you are useing, I have mailboxes with hundreds of thousands of messages in them on XFS and have no problems, but on ext3 I start seeing slowdowns with a bit over ten thousand messages. The usual issue is just the huge number of emails and thus files that accumulate. Creating a fresh replica, body searching, reconstructing, etc all take quite a bit of time because of the large amount of random IOs. Apart from that, everything does actually work ok... The only issue we ever had was with a bboard that our network group sends automated system messages to. Something in their environment went haywire and we ended up with ~1.5 million messages in that bboard. They were unable to find a client that was willing to deal with the folder to be able to clean it up. I was able to connect using imtest and SELECT and FETCH messages without any problems, though. I also recall that replication was broken by this folder, but I don't remember exactly why. alpine and mulberry have no problem with huge numbers of messages. David Lang Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On Tue, 16 Nov 2010, Ciprian wrote: Adam Tauno Williams wrote: I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few hundred thousand messages. Older versions of Outlook (e.g. Office 2003) will choke well before that (I think the limit was around 35.000 on XP SP2 ) also couple that with the local Outlook file store for that IMAP account which used to be limited to 2G. We generally advice our users to avoid going past 20.000 messages in one folder. the nice thing is that this is per-folder, and generally shows up as a slowdown as you get large, so the user can just create a subfolder and move messages in to it to work around the client issues. This should mostly be self-regulating as a result. David Lang Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
ext3 / XFS [Was: Re: Does anyone allow unlimited or extremely large quotas?]
On Tue, 2010-11-16 at 11:25 -0800, David Lang wrote: I don't actually know what sort of problems I'm referring to, hence the question. The big problem I can imagine would be opendir() and readdir() with a huge number of files in a directory, but the cyrus code doesn't appear to do that in a lot of places that would matter to a user (deleting an entire folder, delete sieve scripts, etc) in the course of normal operations. This is depends on what filesystem you are useing, I have mailboxes with hundreds of thousands of messages in them on XFS and have no problems, but on ext3 I start seeing slowdowns with a bit over ten thousand messages. Was dir_index enabled on that ext3 filesystem? Prior to dir-index ext3 was very slow for large folders. dir_index is not enabled by default in ext3. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: ext3 / XFS [Was: Re: Does anyone allow unlimited or extremely large quotas?]
On Tue, 16 Nov 2010, Adam Tauno Williams wrote: On Tue, 2010-11-16 at 11:25 -0800, David Lang wrote: I don't actually know what sort of problems I'm referring to, hence the question. The big problem I can imagine would be opendir() and readdir() with a huge number of files in a directory, but the cyrus code doesn't appear to do that in a lot of places that would matter to a user (deleting an entire folder, delete sieve scripts, etc) in the course of normal operations. This is depends on what filesystem you are useing, I have mailboxes with hundreds of thousands of messages in them on XFS and have no problems, but on ext3 I start seeing slowdowns with a bit over ten thousand messages. Was dir_index enabled on that ext3 filesystem? Prior to dir-index ext3 was very slow for large folders. dir_index is not enabled by default in ext3. yes, even with dir-index I see slowdown on large, busy folders. not as bad as without them, but still there. without dir-index a folder that at one time had 10K files in it becomes unusuably slow forever, with dir-index the slowdown isn't as bad, but it's still there. remember that dir-index only helps for the case where you are looking for a single file, if you are walking the entire directory it has little, if any effect. XFS is slow deleting large numbers of files as noted by others, but delayed expunge sidesteps that issue. David Lang Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On Tue, Nov 16, 2010 at 08:38:49AM -0500, Adam Tauno Williams wrote: On Tue, 2010-11-16 at 14:12 +0100, Rudy Gevaert wrote: On 11/16/2010 12:30 PM, Dave McMurtrie wrote: Good morning, This may be slightly off-topic, so apologies in advance. Is there anyone out there who allows unlimited quota for their users or provides extremely large quotas when asked for? If so, can you describe any problems you've had with this? We have users with 5 GB. Our largest quota's a 4GB; without any issues. Our biggest currently is about 30GB I think. I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few hundred thousand messages. On a 32 bit architecture: we had one folder with over a million messages which was causing processes to run out of virtual memory trying to map the cache file in. This wouldn't be a problem with a 64 bit userland. We haven't seen any problems with them. The only general problem we face is taking backups of a full store. It just takes very long to complete full backups. (Of course that isn't tied to large or no quota.) Nup - full backup would be a problem. We don't do them any more: everything is context aware backups. Bron. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: ext3 / XFS [Was: Re: Does anyone allow unlimited or extremely large quotas?]
This is depends on what filesystem you are useing, I have mailboxes with hundreds of thousands of messages in them on XFS and have no problems, but on ext3 I start seeing slowdowns with a bit over ten thousand messages. Was dir_index enabled on that ext3 filesystem? Prior to dir-index ext3 was very slow for large folders. dir_index is not enabled by default in ext3. FYI our experience at Fastmail 2 years back was that reiserfs still much better than ext3 (even with dir_index) at handling large numbers of files in folders. We tried switching one server to ext3, but after a week or two it was being crushed by load and we switched back to reiserfs. However we've recently found that ext4 is at least as good as reiserfs at handling large directories, so we've started switching everything to ext4 and so far the migration is going well. So don't use ext3, but ext4 is ok. Oh, and we recently setup a spare machine with btrfs and tried replicating a few partitions to it. That wasn't good. Started off promising, but by the time it was 1/3 full, the machine was utterly crawling. Clearly not ready for production yet. Rob Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On 16.11.2010 17:11, Shuvam Misra wrote: I guess Webmail is OT on a Cyrus mailing list, but can't help asking: any suggestions for improving Webmail performance? (Admission: we haven't yet tried imapproxy -- it appears to be a good piece of C which will help things.) You should install imapproxy. Also make sure you do a lot of caching on the webmail side for example if the webmail is programmed with php use apc and cache mailbox listing and message headers if possible. Cache also things on the imap server, if your cyrus version supports it you should try to use status cache and test whether it makes any difference in your environment. Thanks a lot, Leena. :) You've actually tickled some suspicions we have also had, that to deliver good Webmail service to our users, we may have to figure out PHP acceleration. I sometimes wish the world wasn't so desperately pushing the one-size- fits-all paradigm of browser-based apps for each and every thing in sight... all this problem with Webmail is primarily due to using a connectionless run-time system for a connection-oriented service (IMAP). Will look into PHP cacheing and other stuff. Shuvam Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
On 16 Nov 2010, at 10:32, Joseph Brennan wrote: I wish we'd somehow financed a native Cyrus webmail interface, that is not using IMAP but built into Cyrus. I don't think users know how good Cyrus is because they look at it through a weak intermediary. I don't think a Cyrus-specific web interface is the answer to that question. IMP performance is not great, but it's the http paradigm that slows it. Check out roundcube, utilizing AJAX it's way more responsive to the user. Is this a general observation? We used to offer SquirrelMail with our product, and we recently moved to RoundCube because we thought the heavier use of Ajax would make the user experience more responsive. One by one, all our more demanding clients have begun to ask for SqMail back. They say (i) SqMail used to work on the browser of mobile phones but RC doesn't, and (ii) RC is too slow, SqMail was just fine. This second point has caught us by surprise and we are in the process of setting up SqMail again for these customers. Personally we find the SqMail interface quite dated. We were toying with the idea of actually buying and supplying the Tuxedo mail client to some of these customers to see if they'd find it better. Haven't tried IMP. Shuvam Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
Am 16.11.10 19:08, schrieb Wesley Craig: On 16 Nov 2010, at 10:32, Joseph Brennan wrote: I wish we'd somehow financed a native Cyrus webmail interface, that is not using IMAP but built into Cyrus. I don't think users know how good Cyrus is because they look at it through a weak intermediary. I don't think a Cyrus-specific web interface is the answer to that question. IMP performance is not great, but it's the http paradigm that slows it. Check out roundcube, utilizing AJAX it's way more responsive to the user. We started using SOGo here. It just loads a reasonable number of mail items for the index view and continues to load when you scroll down (or up) to get more. As for the FS, we still use Sun aaah Oracle ZFS. Mailboxes with 500,000 messages (postfix mailing list :-) ) are just as SELECTable as empty mailboxes - no difference in speed or access time when retrieving messages from there. The smell of Oracle still gets bitter and bitter compared to Sun, but especially this cookie (zfs) still tastes too well. SOGo is slower (as it has another paradigma as Horde/IMP or Squirrel) but our users seems not to have a problem with it. -- Pascal Gienger Jabber/XMPP/Mail: pascal.gien...@uni-konstanz.de University of Konstanz, IT Services Department (Rechenzentrum) Electronic Communications and Web Services Building V, Room V404, Phone +49 7531 88 5048, Fax +49 7531 88 3739 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: ext3 / XFS [Was: Re: Does anyone allow unlimited or extremely large quotas?]
We use ext4 for more than one year now. Efficient and stable. A good choise 12 spool of 250GB over 10 FC disks using metalun. Dom 2010/11/16 Robert Mueller r...@fastmail.fm This is depends on what filesystem you are useing, I have mailboxes with hundreds of thousands of messages in them on XFS and have no problems, but on ext3 I start seeing slowdowns with a bit over ten thousand messages. Was dir_index enabled on that ext3 filesystem? Prior to dir-index ext3 was very slow for large folders. dir_index is not enabled by default in ext3. FYI our experience at Fastmail 2 years back was that reiserfs still much better than ext3 (even with dir_index) at handling large numbers of files in folders. We tried switching one server to ext3, but after a week or two it was being crushed by load and we switched back to reiserfs. However we've recently found that ext4 is at least as good as reiserfs at handling large directories, so we've started switching everything to ext4 and so far the migration is going well. So don't use ext3, but ext4 is ok. Oh, and we recently setup a spare machine with btrfs and tried replicating a few partitions to it. That wasn't good. Started off promising, but by the time it was 1/3 full, the machine was utterly crawling. Clearly not ready for production yet. Rob Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser.php?uid=lalot Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
[Fwd: Enabled quotas by accident, how do I disable?]
It seems that I've accidentially enabled IMAP quotas on one of my cyrus servers -- users complain that saving mails into certain folders fails with an 'over quota' message. There's indeed a file in the quota directory: bash-3.2$ cat /var/lib/imap/quota/u/user 226215 0 Yesterday evening, I shut down cyrus and removed the /var/lib/imap/quota/u directory and restarted cyrus, but the file has reappeared. Using cyradm, I can confirm that the affected folders are indeed under quota: localhost lqm user/abc/folder1/folder2/folder3/folder4 user STORAGE 220/0 localhost lqm user/abc/folder1/folder2/folder3 localhost I am running Cyrus 2.3.16. The cyrus documentation and web resource does not contain much info about how to _disable_ quotas, so any input from the list is welcomed. - Michael Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: [Fwd: Enabled quotas by accident, how do I disable?]
Michael, I don't know how to disable quota for all user but you can set the quota to -1 or none per user this will give no quota for that user. Edwin. Michael Glad schreef: It seems that I've accidentially enabled IMAP quotas on one of my cyrus servers -- users complain that saving mails into certain folders fails with an 'over quota' message. There's indeed a file in the quota directory: bash-3.2$ cat /var/lib/imap/quota/u/user 226215 0 Yesterday evening, I shut down cyrus and removed the /var/lib/imap/quota/u directory and restarted cyrus, but the file has reappeared. Using cyradm, I can confirm that the affected folders are indeed under quota: localhost lqm user/abc/folder1/folder2/folder3/folder4 user STORAGE 220/0 localhost lqm user/abc/folder1/folder2/folder3 localhost I am running Cyrus 2.3.16. The cyrus documentation and web resource does not contain much info about how to _disable_ quotas, so any input from the list is welcomed. - Michael Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- Edwin Hop ICT Manager AimValley BV Email: e...@aimvalley.nl Phone: 035-6891957 / 06-30327414 Dit e-mailbericht is door Connect Data Solutions gecontroleerd op virussen en spam. This message has been checked on viruses and spam by Connect Data Solutions. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Quotas and disk usage
On Thu, January 14, 2010 8:26 pm, Wil Cooley wrote:
Adam Tauno Williams wrote:
Some earlier versions of Cyrus had quota calculation issues with big
quotas (2GB? 4GB?) or big mailboxes. I think there is a blurb in WMOGAG
about that, and what version fixed it.
It happens with 2.2 crossing the 4GiB range, but not related to what he's
seeing.
bash-3.00$ /opt/cyrus-imap/bin/quota -f user.user1 Quota % Used Used
Root
5242880 63 3303494 user.user1
bash-3.00$ /opt/cyrus-imap/bin/quota -f user.user2 Quota % Used Used
Root
41943040 93 39410366 user.user2
bash-3.00$ du -sh user1 7.6G user1
bash-3.00$ du -sh user2 6.4G user2
Do you have delayed expunge enabled? If so actual disk size may very
well exceed quota values - because the expunged-but-not-yet-expired messages
are still on the disk.
Other possible contributors:
o Squatter indexes (my estimation is that they add 50%) o Directory sizes
themselves o Other cyrus.* metadata
You can use something like this to calculate just the total (sorry, you need
GNU
du, find and xargs):
find `mbpath user.user1` -type f -name \*. -print0 \ |xargs -0 du -cm|awk
'/total/ { tot=tot+$1 } END { print tot MiB }'
You need the awk if there are more files than possible for a single
invocation of du. And you can calculate the metadata overhead by negating
the -name parameter:
find `mbpath user.user1` -type f \! -name \*. -print0 \ |xargs -0 du -cm|awk
'/total/ { tot=tot+$1 } END { print tot MiB }'
On our previous Cyrus server (2.2 on Solaris 9 with UFS) I detected several
multiply-linked message files (sitting in different folders/directories).
Those will be counted twice/thrice/... towards the Cyrus quota but not 'du'.
We were unable to find out how some IMAP clients were capable of having the
server create them.
(This being said, we never invested much effort into it :-)
% find `mbpath user/NN` -type f \! -links 1
On our new server (2.3 on Solaris 10 with ZFS and basic filesystem compression
enabled) there is a (zfs get compressratio) difference of 27 to 28 %
Eric Luyten, Computing Centre VUB/ULB, postmaster.
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Quotas and disk usage
Quoting Eric Luyten eric.luy...@vub.ac.be:
On Thu, January 14, 2010 8:26 pm, Wil Cooley wrote:
Adam Tauno Williams wrote:
Some earlier versions of Cyrus had quota calculation issues with big
quotas (2GB? 4GB?) or big mailboxes. I think there is a blurb in WMOGAG
about that, and what version fixed it.
It happens with 2.2 crossing the 4GiB range, but not related to what he's
seeing.
bash-3.00$ /opt/cyrus-imap/bin/quota -f user.user1 Quota % Used Used
Root
5242880 63 3303494 user.user1
bash-3.00$ /opt/cyrus-imap/bin/quota -f user.user2 Quota % Used Used
Root
41943040 93 39410366 user.user2
bash-3.00$ du -sh user1 7.6G user1
bash-3.00$ du -sh user2 6.4G user2
Do you have delayed expunge enabled? If so actual disk size may very
well exceed quota values - because the
expunged-but-not-yet-expired messages
are still on the disk.
Other possible contributors:
o Squatter indexes (my estimation is that they add 50%) o Directory sizes
themselves o Other cyrus.* metadata
You can use something like this to calculate just the total (sorry, you need
GNU
du, find and xargs):
find `mbpath user.user1` -type f -name \*. -print0 \ |xargs -0 du -cm|awk
'/total/ { tot=tot+$1 } END { print tot MiB }'
You need the awk if there are more files than possible for a single
invocation of du. And you can calculate the metadata overhead by negating
the -name parameter:
find `mbpath user.user1` -type f \! -name \*. -print0 \ |xargs -0 du -cm|awk
'/total/ { tot=tot+$1 } END { print tot MiB }'
On our previous Cyrus server (2.2 on Solaris 9 with UFS) I detected several
multiply-linked message files (sitting in different folders/directories).
Those will be counted twice/thrice/... towards the Cyrus quota but not 'du'.
We were unable to find out how some IMAP clients were capable of having the
server create them.
(This being said, we never invested much effort into it :-)
% find `mbpath user/NN` -type f \! -links 1
It's not the client but the Server doing this. See man imapd.conf
singleinstancestore: 1
If enabled, imapd, lmtpd and nntpd attempt to only write one copy
of a message per partition and create hard links, resulting in a
potentially large disk savings.
On our new server (2.3 on Solaris 10 with ZFS and basic filesystem
compression
enabled) there is a (zfs get compressratio) difference of 27 to 28 %
Eric Luyten, Computing Centre VUB/ULB, postmaster.
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
M.MengeTel.: (49) 7071/29-70316
Universität Tübingen Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung mail:
michael.me...@zdv.uni-tuebingen.de
Wächterstraße 76
72074 Tübingen
smime.p7s
Description: S/MIME Signatur
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Quotas and disk usage
On Fri, January 15, 2010 9:50 am, Michael Menge wrote: Quoting Eric Luyten eric.luy...@vub.ac.be: On our previous Cyrus server (2.2 on Solaris 9 with UFS) I detected several multiply-linked message files (sitting in different folders/directories). Those will be counted twice/thrice/... towards the Cyrus quota but not 'du'. We were unable to find out how some IMAP clients were capable of having the server create them. (This being said, we never invested much effort into it :-) % find `mbpath user/NN` -type f \! -links 1 It's not the client but the Server doing this. See man imapd.conf singleinstancestore: 1 If enabled, imapd, lmtpd and nntpd attempt to only write one copy of a message per partition and create hard links, resulting in a potentially large disk savings. Michael, Due to the way our Postfix interacts with Cyrus I'm pretty sure these doubly-linked message files do not start existing at delivery (lmtp) time, but during IMAP sessions. Some of them (I looked into it earlier today) appear to be the fruits of incorrectly terminated (Mac Mail) user sessions, where the expunge on-exit is not performed. Eric. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Quotas and disk usage
Hi,
We are running Cyrus 2.3.15 on Solaris 10 and recently started enabling
quotas for around 100 users. Certain user accounts show conflicting
information based on the 'quota' command and how much space is taken up
by the filesystem ('du'). We have run 'quota -f' and 'reconstruct -r -f'
numerous times with no change. We sometimes run into accounts with
insanely high numbers which get resolved after a 'reconstruct.' These
are different though, eg.
bash-3.00$ /opt/cyrus-imap/bin/quota -f user.user1
Quota % Used Used Root
5242880 63 3303494 user.user1
bash-3.00$ /opt/cyrus-imap/bin/quota -f user.user2
Quota % Used Used Root
41943040 93 39410366 user.user2
bash-3.00$ du -sh user1
7.6G user1
bash-3.00$ du -sh user2
6.4G user2
As you can see what is reported as 'used' by 'quota' does not match with
what is actually on the file system.
Any thoughts as to what is happening here?
Thank you,
Robert
This e-mail contains information some or all of which may be confidential,
proprietary and/or legally privileged. If an addressing or transmission error
has misdirected this e-mail, please notify the sender by replying to this
e-mail. If you are not the intended recipient you must not use, disclose,
distribute, copy, print or rely on this e-mail.
If you have any queries, or if you no longer wish to receive email in
accordance with the Act on Regulation of the Transmission of Specified
Electronic Mail, please contact the sender of this email.
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Quotas and disk usage
On Thu, 2010-01-14 at 13:25 -0500, Robert Wirstrom wrote:
We are running Cyrus 2.3.15 on Solaris 10 and recently started enabling
quotas for around 100 users. Certain user accounts show conflicting
information based on the 'quota' command and how much space is taken up
by the filesystem ('du').
These won't necessarily correspond.
We have run 'quota -f' and 'reconstruct -r -f'
numerous times with no change. We sometimes run into accounts with
insanely high numbers which get resolved after a 'reconstruct.' These
are different though, eg.
Some earlier versions of Cyrus had quota calculation issues with big
quotas (2GB? 4GB?) or big mailboxes. I think there is a blurb in WMOGAG
about that, and what version fixed it.
bash-3.00$ /opt/cyrus-imap/bin/quota -f user.user1
Quota % Used Used Root
5242880 63 3303494 user.user1
bash-3.00$ /opt/cyrus-imap/bin/quota -f user.user2
Quota % Used Used Root
41943040 93 39410366 user.user2
bash-3.00$ du -sh user1
7.6G user1
bash-3.00$ du -sh user2
6.4G user2
Do you have delayed expunge enabled? If so actual disk size may very
well exceed quota values - because the expunged-but-not-yet-expired
messages are still on the disk.
As you can see what is reported as 'used' by 'quota' does not match with
what is actually on the file system.
Any thoughts as to what is happening here?
--
OpenGroupware developer: awill...@whitemice.org
http://whitemiceconsulting.blogspot.com/
OpenGroupare Cyrus IMAPd documenation @
http://docs.opengroupware.org/Members/whitemice/wmogag/file_view
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Quotas and disk usage
Thanks for your quick reply.
On 01/14/10 01:32 PM, Adam Tauno Williams wrote:
On Thu, 2010-01-14 at 13:25 -0500, Robert Wirstrom wrote:
We are running Cyrus 2.3.15 on Solaris 10 and recently started enabling
quotas for around 100 users. Certain user accounts show conflicting
information based on the 'quota' command and how much space is taken up
by the filesystem ('du').
These won't necessarily correspond.
Understood, but they should be somewhat close in number, right?
We have run 'quota -f' and 'reconstruct -r -f'
numerous times with no change. We sometimes run into accounts with
insanely high numbers which get resolved after a 'reconstruct.' These
are different though, eg.
Some earlier versions of Cyrus had quota calculation issues with big
quotas (2GB? 4GB?) or big mailboxes. I think there is a blurb in WMOGAG
about that, and what version fixed it.
2.3.15 was the latest up until about a month ago. We have no problems
with other users with bigger quotas.
bash-3.00$ /opt/cyrus-imap/bin/quota -f user.user1
Quota % Used Used Root
5242880 63 3303494 user.user1
bash-3.00$ /opt/cyrus-imap/bin/quota -f user.user2
Quota % Used Used Root
41943040 93 39410366 user.user2
bash-3.00$ du -sh user1
7.6G user1
bash-3.00$ du -sh user2
6.4G user2
Do you have delayed expunge enabled? If so actual disk size may very
well exceed quota values - because the expunged-but-not-yet-expired
messages are still on the disk.
Delayed expunge is not enabled.
As you can see what is reported as 'used' by 'quota' does not match with
what is actually on the file system.
Any thoughts as to what is happening here?
This e-mail contains information some or all of which may be confidential,
proprietary and/or legally privileged. If an addressing or transmission error
has misdirected this e-mail, please notify the sender by replying to this
e-mail. If you are not the intended recipient you must not use, disclose,
distribute, copy, print or rely on this e-mail.
If you have any queries, or if you no longer wish to receive email in
accordance with the Act on Regulation of the Transmission of Specified
Electronic Mail, please contact the sender of this email.
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Quotas and disk usage
Adam Tauno Williams wrote:
Some earlier versions of Cyrus had quota calculation issues with big
quotas (2GB? 4GB?) or big mailboxes. I think there is a blurb in WMOGAG
about that, and what version fixed it.
It happens with 2.2 crossing the 4GiB range, but not related to what he's
seeing.
bash-3.00$ /opt/cyrus-imap/bin/quota -f user.user1
Quota % Used Used Root
5242880 63 3303494 user.user1
bash-3.00$ /opt/cyrus-imap/bin/quota -f user.user2
Quota % Used Used Root
41943040 93 39410366 user.user2
bash-3.00$ du -sh user1
7.6G user1
bash-3.00$ du -sh user2
6.4G user2
Do you have delayed expunge enabled? If so actual disk size may very
well exceed quota values - because the expunged-but-not-yet-expired
messages are still on the disk.
Other possible contributors:
o Squatter indexes (my estimation is that they add 50%)
o Directory sizes themselves
o Other cyrus.* metadata
You can use something like this to calculate just the total (sorry, you need GNU
du, find and xargs):
find `mbpath user.user1` -type f -name \*. -print0 \
|xargs -0 du -cm|awk '/total/ { tot=tot+$1 } END { print tot MiB }'
You need the awk if there are more files than possible for a single invocation
of du. And you can calculate the metadata overhead by negating the -name
parameter:
find `mbpath user.user1` -type f \! -name \*. -print0 \
|xargs -0 du -cm|awk '/total/ { tot=tot+$1 } END { print tot MiB }'
Wil
signature.asc
Description: OpenPGP digital signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Were is quotas information stored
Hello everyone, As it seems impossible to access users accounts of many domains in cyradm, As it seems impossible to change the quota of a user in another domain, I would like to know where is stored the quotas for a specific user account ? For example I have user dbuche...@mydomain3.com, with path : /var/spool/imap/domain/m/mydomain3.com/d/user/dbucherml/ Where is the quota information stored, and how could I set unlimited quotas ? That's not so easy to say because it depends on your configuration. In the default config you should find the quota flat databases somewhere under $CONFIGDIR/quota. While it's usually a bad idea to touch those files by hand you should be able to 1) shutdown cyrus-imapd 2) remove the users quota file 3) startup cyrus-imapd until you found out how to access quota the correct way to cyradm. Simon Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Were is quotas information stored
Hello everyone, As it seems impossible to access users accounts of many domains in cyradm, As it seems impossible to change the quota of a user in another domain, I would like to know where is stored the quotas for a specific user account ? For example I have user dbuche...@mydomain3.com, with path : /var/spool/imap/domain/m/mydomain3.com/d/user/dbucherml/ Where is the quota information stored, and how could I set unlimited quotas ? Thanks a lot for any help... Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Disabling quotas in cyrus
Hello all. We are doing some substantial number of moves of users between backends here, and have been running into problems with quotas.db. We are currently running 2.3.8 as our version of cyrus. The problem is that we seem to run into lock issues with quotas.db (nothing else yet, thankfully!) where errors arise, and in some cases it brings cyrus to a halt. We have to then grab a last known good quotas.db file (either from a backup or a export that we import) and then restart with that. Then we do a quota -f to update, and await the users that aren't in the file (since they were moved after that last known good copy) to pop up on the list so we can fix their quota (as it gives them 0 by default). The problems with this are those users start bouncing mail, it takes a lot of manual intervention, downtime on the mail server, etc. So, what we were thinking of is disabling quotas on the servers we're moving to, so that this problem is mitigated and our moves can continue at a good pace. Currently we've just been sticking to a single batch script that only does one xfer at a time, as that seems to avoid any major problems. (we still see it occasionally, but it seems to recover on its own) We would like to run multiples, but it seems to really start to have problems with locking on the multiple xfers. We haven't noticed any problems on the from boxes with moves. Also, once disabled, we'd then need to be able to re-enable and set quotas for everyone at a future time. (Hopefully without a manual one at a time process) Is there any semi-easy solution to this? If not, we'll stick to what we're doing, but wanted to ask. Also, this problem doesn't seem to happen in normal functions of our servers (we hadn't before run into it until these xfers) If anyone has further questions for me, I'll answer as best I can. Sample log entries below for the curious. Sorry if this is a RTFM question, I just haven't seen anything about it in my searches. Thanks! Tim Champ UMBC DoIT Unix Infrastructure Team Logs: $DATE MAILSERVER imap[14262]: [ID 335833 local6.error] DBERROR: error fetching user.$USER.$FOLDER: cyrusdb error $DATE MAILSERVER imap[14262]: [ID 602473 local6.error] IOERROR: lock_shared /$SERVER/data/config/quotas.db: Bad file number --- This sometimes resolves itself fine, but other times it causes cyrus to quit functioning. If you attempt to restart cyrus, it flakes out with quotas.db being invalid. This log blurb is during the move of $USER. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
how to replicate quotas in murder replicated mode?
hello everybody! i am testing the replicated config of murder but i can't see how to replicate quotas info. it is posible? regards. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Question about managing over-quotas mailboxes and deleted mails
Hi, i want to ask is there some way to solve the problem with users that have over-quota in their mailboxes. I'm not sure if they delete there messages with purge delete or not. I suppuse if they don't delete the messages then mailbox going to be over quota. Is there some way to understand if some message is marked like deleted, or not? We can increase the quota, but i want to understand what is the real reason for over quota for these users. Thanks in advanced! Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
cyrus/imap set quotas on cyrus/imap mailboxes.
Hi List I have the notes for cyrus/imap set quotas something like this from a previous employer. Thus I would like clarificartion or url to a good tutorial or howto. Use /cyradm/ to set quota to 50mb localhost setquota user.joey 50.000 quota:5.000 localhost lq user.joey STORAGE 0/5.000 (0%) add *quotawarn* */etc/imapd.conf* to Warn Users vi and add the following below:: *quotawarn: 90* Please can someone suggest a good list or is it just the case of joining cyruss/imap as I want to read and digest setting up cyrus/imap set quotas to get a a handle on it. Cheers Chuck -- Chuck Amadi ROK Corporation Limited Ty ROK, Dyffryn Business Park, Llantwit Major Road, Llandow, Vale Of Glamorgan. CF71 7PY Tel: 01446 795 839 Fax: 01446 794 994 International Tel: +44 1446 795 839 email: [EMAIL PROTECTED] This email is confidential to the addressee only. If you do not believe that you are the intended recipient, do not pass it on or copy it in any way. Please delete it immediately. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus/imap set quotas on cyrus/imap mailboxes.
Hi there on this list too, I think oreilly's managing imap has a public chapter on managing cyrus which could be useful. You can find it here: http://www.oreilly.com/catalog/mimap/chapter/ch09.html Hi List I have the notes for cyrus/imap set quotas something like this from a previous employer. Thus I would like clarificartion or url to a good tutorial or howto. Use /cyradm/ to set quota to 50mb localhost setquota user.joey 50.000 quota:5.000 localhost lq user.joey STORAGE 0/5.000 (0%) add *quotawarn* */etc/imapd.conf* to Warn Users vi and add the following below:: *quotawarn: 90* Please can someone suggest a good list or is it just the case of joining cyruss/imap as I want to read and digest setting up cyrus/imap set quotas to get a a handle on it. Cheers Chuck -- Chuck Amadi ROK Corporation Limited Ty ROK, Dyffryn Business Park, Llantwit Major Road, Llandow, Vale Of Glamorgan. CF71 7PY Tel: 01446 795 839 Fax: 01446 794 994 International Tel: +44 1446 795 839 email: [EMAIL PROTECTED] This email is confidential to the addressee only. If you do not believe that you are the intended recipient, do not pass it on or copy it in any way. Please delete it immediately. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- Aleksandr Stankevic Unix system administrator Mobile: +370 650 28747 ICQ: 214480900 Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus/imap set quotas on cyrus/imap mailboxes.
Aleksandr Stankevic wrote: Hi there on this list too, I think oreilly's managing imap has a public chapter on managing cyrus which could be useful. You can find it here: http://www.oreilly.com/catalog/mimap/chapter/ch09.html Hi List I have the notes for cyrus/imap set quotas something like this from a previous employer. Thus I would like clarificartion or url to a good tutorial or howto. Use /cyradm/ to set quota to 50mb localhost setquota user.joey 50.000 quota:5.000 localhost lq user.joey STORAGE 0/5.000 (0%) add *quotawarn* */etc/imapd.conf* to Warn Users vi and add the following below:: *quotawarn: 90* Please can someone suggest a good list or is it just the case of joining cyruss/imap as I want to read and digest setting up cyrus/imap set quotas to get a a handle on it. Cheers Chuck -- Chuck Amadi ROK Corporation Limited Ty ROK, Dyffryn Business Park, Llantwit Major Road, Llandow, Vale Of Glamorgan. CF71 7PY Tel: 01446 795 839 Fax: 01446 794 994 International Tel: +44 1446 795 839 email: [EMAIL PROTECTED] This email is confidential to the addressee only. If you do not believe that you are the intended recipient, do not pass it on or copy it in any way. Please delete it immediately. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html Hi Aleksandr Excellent . Cheers -- Chuck Amadi ROK Corporation Limited Ty ROK, Dyffryn Business Park, Llantwit Major Road, Llandow, Vale Of Glamorgan. CF71 7PY Tel: 01446 795 839 Fax: 01446 794 994 International Tel: +44 1446 795 839 email: [EMAIL PROTECTED] This email is confidential to the addressee only. If you do not believe that you are the intended recipient, do not pass it on or copy it in any way. Please delete it immediately. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: per-user quotas
Quoting Philippe Trolliet [EMAIL PROTECTED]: Hi, in the log i see following message: ... Mar 13 12:17:41 mail lmtpunix[10500]: verify_user(novaware.de!user.p^trolliet) failed: Over quota Mar 13 12:17:41 mail postfix/pipe[10689]: E644F5E44F9: to=[EMAIL PROTECTED], relay=cyrus, delay=1, status=deferred (temporary failure) ... in one of the mailinglist archives i read sth about the error codes. my problem is that the message is not bounced and so the sender is not notified. it is stuck somewhere in the postfix-queue. i have virtual users and domains and i´m using per-user quotas stored in a mysql-db. i can set the quota for every single user. what do i have to configure that cyrus bounces the message over the lmtp if a quota exceeds? in the archives somebody said sth about the error codes too. how can i do this in the cyrus configuration? postfix will also bounce the messages on temporary failure if it could not delive the message for some time (maximal_queue_lifetime default: 5d) is there any way to tell cyrus to read the per-user quota settings from the mysql-db like in dovecot? or is this just done in postfix? you could write an perl-script which reads the data form the mysql-db and sets the quota in cyrus M.Menge Tel.: (49) 7071/29-70316 Universitaet Tuebingen Fax.: (49) 7071/29-5912 Zentrum fuer Datenverarbeitung mail: [EMAIL PROTECTED] Waechterstrasse 76 72074 Tuebingen smime.p7s Description: S/MIME krytographische Unterschrift Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: per-user quotas
Andrew Morgan wrote: On Tue, 13 Mar 2007, Philippe Trolliet wrote: you are my hero. ;) thank you. finally i´m able to test it on thursday. but it seems that cyrus knows in some way which quota the specific user has. is it possible that cyrus communicates with postfix and asks postfix about the quota of the recipient? i´ve set autocreatequota to 40 MB and in postfix i´ve set the quota for the test-account to 10MB. i send some mail with big attachments to this test account. the mailbox of the user has a size of something over 10MB now and doesn´t accept any mail. the messages aren´t delivered and deferred now. so i think there is a way cyrus talks to postfix about per-user quotas. I don't know what setting a quota in postfix does... You can view the quota in Cyrus by running cyradm and executing the command lq user.username. Andy i have these settings in postfix: /etc/postfix/main.cf: ... virtual_create_maildirsize = yes virtual_mailbox_extended = yes virtual_mailbox_limit_maps = proxy:mysql:/etc/postfix/mysql-virtual_mailbox_limit_maps.cf virtual_mailbox_limit_override = yes virtual_maildir_limit_message = Sorry, the user's maildir has overdrawn his diskspace quota, please try again later. virtual_overquota_bounce = yes ... /etc/postfix/mysql-virtual_mailbox_limit_maps.cf: user = user password = pw hosts = localhost dbname = postfixadmin query = SELECT quota FROM mailbox WHERE username='%s' in my cyrus config i have autocreatequota enabled with 40 MB. with postfixadmin i´ve set the user quota for my test account to 10MB and seems that cyrus knows in some way about this quota. but i don´t know how and why. regards philippe Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
per-user quotas
hi, i´m using postfix and cyrus-imapd with virtual users and domains. is it possible to get imapd to work with per-user quotas? i saw that there are some patches out there for postfix. but the developers of postfix recommend filesystem quotas. i´m asking this because dovecot has the ability to use per-user quotas. regards philippe -- I am using the free version of SPAMfighter for private users. It has removed 4012 spam emails to date. Paying users do not have this message in their emails. Get the free SPAMfighter here: http://www.spamfighter.com/len Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: per-user quotas
Hi what do you mean with per-user quotas? Cyrus has its own quota system, where you can set the quota for each user, or even on subfolders. In case of quota exceeding cyrus will reject the message in the lmtp protocol with a 4xx or a 5xx error, depending of how you configure cyrus. As far as i know, there is no need of filesystem based quota or patching postfix. Quoting Philippe Trolliet [EMAIL PROTECTED]: hi, i´m using postfix and cyrus-imapd with virtual users and domains. is it possible to get imapd to work with per-user quotas? i saw that there are some patches out there for postfix. but the developers of postfix recommend filesystem quotas. i´m asking this because dovecot has the ability to use per-user quotas. regards philippe -- I am using the free version of SPAMfighter for private users. It has removed 4012 spam emails to date. Paying users do not have this message in their emails. Get the free SPAMfighter here: http://www.spamfighter.com/len Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html M.Menge Tel.: (49) 7071/29-70316 Universitaet Tuebingen Fax.: (49) 7071/29-5912 Zentrum fuer Datenverarbeitung mail: [EMAIL PROTECTED] Waechterstrasse 76 72074 Tuebingen smime.p7s Description: S/MIME krytographische Unterschrift Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: per-user quotas
Hi, in the log i see following message: ... Mar 13 12:17:41 mail lmtpunix[10500]: verify_user(novaware.de!user.p^trolliet) failed: Over quota Mar 13 12:17:41 mail postfix/pipe[10689]: E644F5E44F9: to=[EMAIL PROTECTED], relay=cyrus, delay=1, status=deferred (temporary failure) ... in one of the mailinglist archives i read sth about the error codes. my problem is that the message is not bounced and so the sender is not notified. it is stuck somewhere in the postfix-queue. i have virtual users and domains and i´m using per-user quotas stored in a mysql-db. i can set the quota for every single user. what do i have to configure that cyrus bounces the message over the lmtp if a quota exceeds? in the archives somebody said sth about the error codes too. how can i do this in the cyrus configuration? is there any way to tell cyrus to read the per-user quota settings from the mysql-db like in dovecot? or is this just done in postfix? Michael Menge wrote: Hi what do you mean with per-user quotas? Cyrus has its own quota system, where you can set the quota for each user, or even on subfolders. In case of quota exceeding cyrus will reject the message in the lmtp protocol with a 4xx or a 5xx error, depending of how you configure cyrus. As far as i know, there is no need of filesystem based quota or patching postfix. Quoting Philippe Trolliet [EMAIL PROTECTED]: hi, i´m using postfix and cyrus-imapd with virtual users and domains. is it possible to get imapd to work with per-user quotas? i saw that there are some patches out there for postfix. but the developers of postfix recommend filesystem quotas. i´m asking this because dovecot has the ability to use per-user quotas. regards philippe -- I am using the free version of SPAMfighter for private users. It has removed 4012 spam emails to date. Paying users do not have this message in their emails. Get the free SPAMfighter here: http://www.spamfighter.com/len Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html M.Menge Tel.: (49) 7071/29-70316 Universitaet Tuebingen Fax.: (49) 7071/29-5912 Zentrum fuer Datenverarbeitung mail: [EMAIL PROTECTED] Waechterstrasse 76 72074 Tuebingen Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: per-user quotas
On Tue, 13 Mar 2007, Philippe Trolliet wrote: Hi, in the log i see following message: ... Mar 13 12:17:41 mail lmtpunix[10500]: verify_user(novaware.de!user.p^trolliet) failed: Over quota Mar 13 12:17:41 mail postfix/pipe[10689]: E644F5E44F9: to=[EMAIL PROTECTED], relay=cyrus, delay=1, status=deferred (temporary failure) ... in one of the mailinglist archives i read sth about the error codes. my problem is that the message is not bounced and so the sender is not notified. it is stuck somewhere in the postfix-queue. i have virtual users and domains and i´m using per-user quotas stored in a mysql-db. i can set the quota for every single user. what do i have to configure that cyrus bounces the message over the lmtp if a quota exceeds? in the archives somebody said sth about the error codes too. how can i do this in the cyrus configuration? In /etc/imapd.conf: lmtp_over_quota_perm_failure: 1 is there any way to tell cyrus to read the per-user quota settings from the mysql-db like in dovecot? or is this just done in postfix? Cyrus stores quotas in its own quota files. I'm not aware of any way to read a quota from mysql. Andy Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: per-user quotas
you are my hero. ;) thank you. finally i´m able to test it on thursday. but it seems that cyrus knows in some way which quota the specific user has. is it possible that cyrus communicates with postfix and asks postfix about the quota of the recipient? i´ve set autocreatequota to 40 MB and in postfix i´ve set the quota for the test-account to 10MB. i send some mail with big attachments to this test account. the mailbox of the user has a size of something over 10MB now and doesn´t accept any mail. the messages aren´t delivered and deferred now. so i think there is a way cyrus talks to postfix about per-user quotas. Andrew Morgan wrote: On Tue, 13 Mar 2007, Philippe Trolliet wrote: Hi, in the log i see following message: ... Mar 13 12:17:41 mail lmtpunix[10500]: verify_user(novaware.de!user.p^trolliet) failed: Over quota Mar 13 12:17:41 mail postfix/pipe[10689]: E644F5E44F9: to=[EMAIL PROTECTED], relay=cyrus, delay=1, status=deferred (temporary failure) ... in one of the mailinglist archives i read sth about the error codes. my problem is that the message is not bounced and so the sender is not notified. it is stuck somewhere in the postfix-queue. i have virtual users and domains and i´m using per-user quotas stored in a mysql-db. i can set the quota for every single user. what do i have to configure that cyrus bounces the message over the lmtp if a quota exceeds? in the archives somebody said sth about the error codes too. how can i do this in the cyrus configuration? In /etc/imapd.conf: lmtp_over_quota_perm_failure: 1 is there any way to tell cyrus to read the per-user quota settings from the mysql-db like in dovecot? or is this just done in postfix? Cyrus stores quotas in its own quota files. I'm not aware of any way to read a quota from mysql. Andy Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: per-user quotas
On Tue, 13 Mar 2007, Philippe Trolliet wrote: you are my hero. ;) thank you. finally i´m able to test it on thursday. but it seems that cyrus knows in some way which quota the specific user has. is it possible that cyrus communicates with postfix and asks postfix about the quota of the recipient? i´ve set autocreatequota to 40 MB and in postfix i´ve set the quota for the test-account to 10MB. i send some mail with big attachments to this test account. the mailbox of the user has a size of something over 10MB now and doesn´t accept any mail. the messages aren´t delivered and deferred now. so i think there is a way cyrus talks to postfix about per-user quotas. I don't know what setting a quota in postfix does... You can view the quota in Cyrus by running cyradm and executing the command lq user.username. Andy Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
64-bit quotas with 2.2.13
I tried to apply the patch for 2.2.12 to get quotas beyond 2 GiB, but it doesn't apply very well. has anyone ported it to 2.2.13? -- thanks, Kjetil T. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: What exactly *IS* the status of 2GB quotas? [WAS Re: ****Re: Odd quota problem]
Scott, just for some simulated testing I installed Cyrus 2.3 on a test box and loaded up a mailbox to over 10GB (hundreds of thousands of tiny messages). The mailbox was still responsive, however the squat process would take a long time to run and would lock the mailbox for long periods while it did so. This is on old hardware (PentiumIII with a 15GB IDE drive), but new software (Fedora Core 5, Cyrus 2.3, Postfix 2.2 or 2.3, JFS). YMMV. -Blake Daniel Eckl wrote: Hi Scott! You don't need Ken here. You need google. :) https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=1212 https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=2690 Best, Daniel Scott Adkins schrieb: --On Saturday, August 05, 2006 6:55 PM +0200 Simon Matter [EMAIL PROTECTED] wrote: Okay, I would like to hear specifically from Ken on this one... I asked the question myself and go an answer suggesting the limit is not 2GB, but 2TB. I saw a posting from Ken earlier on suggesting something similar. In a nutshell, it looks like we can set quotas to any size (even larger than 4GB) because the quote limit is defined in Kilobytes and note bytes. However, the real problem is the tracking of the actual space used by the user, which is done in bytes... so, even though you can set the quota larger, the 2GB limit still gets hit because the actual usage counter overflows. Is this how it really is, or am I barking up the wrong tree? There also seem to be some third-party patches out there (Simon, etc) that implement a large quota patch, which gets around this problem. It looks like it is actually a 64-bit quota patch, which allows Cyrus to track how much disk space is used to greater than 4GB. Again, am I interpreting this correctly? So, to wrap up this discussion once and for all: What is the actual quota limitation for Cyrus 2.2 and for Cyrus 2.3? (are they different? did the third party patches make it into the 2.3 versions of Cyrus?) Does the large quota patch require a 64-bit machine/os in order to permit quotas larger than 2GB? Thanks, Scott On Fri, 2006-08-04 at 17:34 +0200, Daniel Eckl wrote: Perhaps I didn't describe the max quota limit problem good enough: When your system is affected by this problem and you set a quota greater than 2 GB, it results in running mad. In this case it will report over quota way before 2GB. Setting quota back to 2GB (or some bytes below), the user will be able to use this 2GB w/o problems again. So your symptoms point exactly to this problem. In this case, the only way to have more space as 2GB is to remove quota like it has been suggested here already. are there any versions of cyrus 2.2.x that are fixed to allow quota's over 2Gb or is that fix only available for 2.3.x ? I have large quota patches in my 2.2.12 rpms which seem to work fine. Simon Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: What exactly *IS* the status of 2GB quotas? [WAS Re: ****Re: Odd quota problem]
--On Saturday, August 05, 2006 6:55 PM +0200 Simon Matter [EMAIL PROTECTED] wrote: Okay, I would like to hear specifically from Ken on this one... I asked the question myself and go an answer suggesting the limit is not 2GB, but 2TB. I saw a posting from Ken earlier on suggesting something similar. With 2.2 it was 2GB, with 2.3 it may be 2TB or something, just usually large enough. In a nutshell, it looks like we can set quotas to any size (even larger than 4GB) because the quote limit is defined in Kilobytes and note bytes. However, the real problem is the tracking of the actual space used by the user, which is done in bytes... so, even though you can set the quota larger, the 2GB limit still gets hit because the actual usage counter overflows. Is this how it really is, or am I barking up the wrong tree? There also seem to be some third-party patches out there (Simon, etc) that implement a large quota patch, which gets around this problem. It looks like it is actually a 64-bit quota patch, which allows Cyrus to track how much disk space is used to greater than 4GB. Again, am I interpreting this correctly? So, to wrap up this discussion once and for all: What is the actual quota limitation for Cyrus 2.2 and for Cyrus 2.3? (are they different? did the third party patches make it into the 2.3 versions of Cyrus?) 64-bit quota has been made in 2.3 and backported to 2.2. They work the same way. I don't know what your problem is but maybe you have to update quota with quota -f after switching your mailspool to the 64bit quota thing. Does the large quota patch require a 64-bit machine/os in order to permit quotas larger than 2GB? No, it works fine on 32bit cpu/os. Simon Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: What exactly *IS* the status of 2GB quotas? [WAS Re: ****Re: Odd quota problem]
--On Wednesday, August 09, 2006 7:10 PM +0200 Daniel Eckl [EMAIL PROTECTED] wrote: https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=1212 https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=2690 Thanks! --On Wednesday, August 09, 2006 8:36 PM +0200 Simon Matter [EMAIL PROTECTED] wrote: With 2.2 it was 2GB, with 2.3 it may be 2TB or something, just usually large enough. Great! 64-bit quota has been made in 2.3 and backported to 2.2. They work the same way. I don't know what your problem is but maybe you have to update quota with quota -f after switching your mailspool to the 64bit quota thing. Heh... Did I say what my problem is? :) No problem... no problem... I just wanted to know the definitive answer, which you guys gave me! Does the large quota patch require a 64-bit machine/os in order to permit quotas larger than 2GB? No, it works fine on 32bit cpu/os. Thank you both for your quick answers. FYI, we are running on a version older than 2.2.12. However, we will be likely upgrading to Linux and the latest version of Cyrus maybe in December. It is unlikely we will patch our software right now to accomodate quotas larger than 2GB, but maybe it won't be up to me on that... Only 1 user is requesting a larger quota than that right now. Scott -- +---+ Scott W. Adkinshttp://www.cns.ohiou.edu/~sadkins/ UNIX Systems Engineer mailto:[EMAIL PROTECTED] ICQ 7626282 Work (740)593-9478 Fax (740)593-1944 +---+ PGP Public Key available at http://www.cns.ohiou.edu/~sadkins/pgp/ pgpnKiI8k9BKe.pgp Description: PGP signature Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: What exactly *IS* the status of 2GB quotas? [WAS Re: ****Re: Odd quota problem]
--On Wednesday, August 09, 2006 7:10 PM +0200 Daniel Eckl [EMAIL PROTECTED] wrote: https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=1212 https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=2690 Thanks! --On Wednesday, August 09, 2006 8:36 PM +0200 Simon Matter [EMAIL PROTECTED] wrote: With 2.2 it was 2GB, with 2.3 it may be 2TB or something, just usually large enough. Great! 64-bit quota has been made in 2.3 and backported to 2.2. They work the same way. I don't know what your problem is but maybe you have to update quota with quota -f after switching your mailspool to the 64bit quota thing. Heh... Did I say what my problem is? :) No problem... no problem... I just wanted to know the definitive answer, which you guys gave me! Does the large quota patch require a 64-bit machine/os in order to permit quotas larger than 2GB? No, it works fine on 32bit cpu/os. Thank you both for your quick answers. FYI, we are running on a version older than 2.2.12. However, we will be likely upgrading to Linux and the latest version of Cyrus maybe in December. It is unlikely we will patch our software right now to accomodate quotas larger than 2GB, but maybe it won't be up to me on that... Only 1 user is requesting a larger quota than that right now. Okay, just to make it clear again, 2.2.12 does not include the 64bit quota patches but I included the patches in my 2.2.12 rpms. Simon Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Problem with quotas larger than 4 GByte
Hello, I have trouble with a power user mailbox, which currently runs near at 4GByte. I bumped up the quota to 6 Gbyte, but deliver fails then: Mar 29 10:08:37 mail lmtpunix[2968]: verify_user(user.dietrich^habs) failed: Over quota cyrus-admin shows the following: localhost.localdomain lq user/dietrich.habs STORAGE 3722027/600 (62.03378%) localhost.localdomain So this looks like lmtpunix has an problem with number over 32bit! I run the following cyrus version: 2.2.12 with autocreate and autosieve patches As an short time solution I removed quota for this user, but I fear that he will run into storage limits. Sincerly, Klaus Steinberger -- Klaus Steinberger Maier-Leibnitz Labor Phone: (+49 89)289 14287 Am Coulombwall 6, D-85748 Garching, Germany FAX: (+49 89)289 14280 EMail: [EMAIL PROTECTED] URL: http://www.physik.uni-muenchen.de/~k2/ In a world without Walls and Fences, who needs Windows and Gates Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Problem with quotas larger than 4 GByte
Klaus Steinberger [EMAIL PROTECTED] writes: I have trouble with a power user mailbox, which currently runs near at 4GByte. I bumped up the quota to 6 Gbyte, but deliver fails then: Yes. https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=2690 Darrell Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Problem with quotas larger than 4 GByte
--On 29. März 2006 10:42:31 +0200 Klaus Steinberger [EMAIL PROTECTED] wrote: I have trouble with a power user mailbox, which currently runs near at 4GByte. I bumped up the quota to 6 Gbyte, but deliver fails then: Mar 29 10:08:37 mail lmtpunix[2968]: verify_user(user.dietrich^habs) failed: Over quota Only Cyrus 2.3 supports quotas of that size. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - Tel. +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpGI9tgJ7p0J.pgp Description: PGP signature Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Problem with quotas larger than 4 GByte
--On 29. März 2006 10:42:31 +0200 Klaus Steinberger [EMAIL PROTECTED] wrote: I have trouble with a power user mailbox, which currently runs near at 4GByte. I bumped up the quota to 6 Gbyte, but deliver fails then: Mar 29 10:08:37 mail lmtpunix[2968]: verify_user(user.dietrich^habs) failed: Over quota Only Cyrus 2.3 supports quotas of that size. Or my latest 2.2 rpms which are patched for it. Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Strange quotas
Hello, I'm running into a strange problems using quotas under Cyrus 2.2 (haven't tried 2.3 yet). I'm using virtual domains and I've defined domain-wise quota, like sq @domain.tld 10 and under @domain.tld, I have certain users who have quotas too, like sq user/[EMAIL PROTECTED] 5000 and some others who have no quotas. The users without quotas have the quotaroot updated when they get a new mail, but the users with quotas only get their own quotas updated when they get a message; the quotaroot isn't updated. Am I missing something, is this a normal thing ? I would have expected something like domain (quota ) +--- user1 (quota Y) +--- user2 (no quota) +--- user3 (quota Z) +--- user4 (no quota) With quota the number that u1+u2+u3+u4 can't go beyond, but it appears that this particular number is right now +Y+Z, which is obviously not what I expected... Thanks, Arnaud. Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Removing quotas in 2.3.1 broken?
Greets. I'm having problems removing quotas in Cyrus 2.3.1. I'm using Simon's 2.3.1-2 RPM. It looks like the command attempts to remove the mailbox quota but instead ends up doing something else. What I'm not sure of so lots of logs and details provided: Before starting I run quota -f to make sure everything is sane: bash-3.00$ /usr/lib/cyrus-imapd/quota -f Quota % Used Used Root 6000 user.drfickle 600 4120 user.lnxgeek 6000 user.osubeav 8007 user.scottrus Using cyradm I look at the existing quota for user.scottrus, remove it, then look at it again. [EMAIL PROTECTED] users]# cyradm --user cyrus localhost Password: localhost.localdomain lq user.scottrus STORAGE 7/80 (0.000875%) localhost.localdomain sq user.scottrus none remove quota localhost.localdomain lq user.scottrus / localhost.localdomain quit The last listquota command above returns odd results. I would expect to see blank line as with mailboxes that have been created without a quota limit set. Looking at the trace though everything looks okay: 11398677184 RLIST 1139867718* LIST (\Noselect) . 4 OK Completed (0.000 secs 0 calls) 11398677225 GETQUOTA user.scottrus 1139867722* QUOTA user.scottrus (STORAGE 7 80) 5 OK Completed 11398677276 SETQUOTA user.scottrus () 11398677276 OK Completed 11398677307 GETQUOTA user.scottrus 1139867730* QUOTA user.scottrus () 7 OK Completed However the quota command still shows some odd results for user.scottrus after attempting to remove the quota: bash-3.00$ /usr/lib/cyrus-imapd/quota -f Quota % Used Used Root 6000 user.drfickle 600 4120 user.lnxgeek 6000 user.osubeav 7 user.scottrus From what I understand everything is okay here. For user.scottrus only the size of the mailbox is shown, no quota is shown because it is set to -1 in the quota db. I think the problem is that the quota file (assuming you are using quota legacy db) is not removed when setting quota to none, instead the quota is set to -1. quota -f reports for every used who has a quota file, but correctly shows no quota for it. What exactly do you consider broken here? Simon Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Removing quotas in 2.3.1 broken?
Hi Scott, --On 13. Februar 2006 17:07:16 -0500 Scott Russell [EMAIL PROTECTED] wrote: Using cyradm I look at the existing quota for user.scottrus, remove it, then look at it again. [EMAIL PROTECTED] users]# cyradm --user cyrus localhost Password: localhost.localdomain lq user.scottrus STORAGE 7/80 (0.000875%) localhost.localdomain sq user.scottrus none remove quota I didn't even know you could do it like that. I always use sq user.scottrus 0 localhost.localdomain lq user.scottrus / localhost.localdomain quit The last listquota command above returns odd results. I would expect to see blank line as with mailboxes that have been created without a quota limit set. I can confirm your observation. But the behavior is the same under 2.2.12. I guess that having a 0 quota and setting the quota to none have the same effect. And apparently you can't go back to the state you had before you ever set a quota. Cheers, Sebastian Hagedorn -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - Tel. +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgptTvRD8sASW.pgp Description: PGP signature Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Removing quotas in 2.3.1 broken?
Hi Scott, --On 13. Februar 2006 17:07:16 -0500 Scott Russell [EMAIL PROTECTED] wrote: Using cyradm I look at the existing quota for user.scottrus, remove it, then look at it again. [EMAIL PROTECTED] users]# cyradm --user cyrus localhost Password: localhost.localdomain lq user.scottrus STORAGE 7/80 (0.000875%) localhost.localdomain sq user.scottrus none remove quota I didn't even know you could do it like that. I always use sq user.scottrus 0 localhost.localdomain lq user.scottrus / localhost.localdomain quit The last listquota command above returns odd results. I would expect to see blank line as with mailboxes that have been created without a quota limit set. I can confirm your observation. But the behavior is the same under 2.2.12. I guess that having a 0 quota and setting the quota to none have the same effect. And apparently you can't go back to the state you had before you ever set a quota. At least with quotalegacy you can do it the dirty way by simply removing the quota file for the user. Simon Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Removing quotas in 2.3.1 broken?
Scott Russell wrote: Greets. I'm having problems removing quotas in Cyrus 2.3.1. I'm using Simon's 2.3.1-2 RPM. It looks like the command attempts to remove the mailbox quota but instead ends up doing something else. What I'm not sure of so lots of logs and details provided: Hello, As Simon already said the quota removal works a bit differently in cyrus imap. You can read more info on this url : http://email.uoa.gr/projects/cyrus/quota-patches/rmquota/ The rmquota patch exists for cyrus 2.2.12. It hasn't been ported yet (officialy at least) to 2.3.1 since there wasn't much demand. If there is more demand we will port it asap to 2.3.1. (the 'auto' patches are by far more popular). Best Regards, Aristotelis Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Removing quotas in 2.3.1 broken?
Scott Russell wrote: Greets. I'm having problems removing quotas in Cyrus 2.3.1. I'm using Simon's 2.3.1-2 RPM. It looks like the command attempts to remove the mailbox quota but instead ends up doing something else. What I'm not sure of so lots of logs and details provided: Hello, As Simon already said the quota removal works a bit differently in cyrus imap. You can read more info on this url : http://email.uoa.gr/projects/cyrus/quota-patches/rmquota/ The rmquota patch exists for cyrus 2.2.12. It hasn't been ported yet (officialy at least) to 2.3.1 since there wasn't much demand. If there is more demand we will port it asap to 2.3.1. (the 'auto' patches are by far more popular). I have ported the rmquota patch to 2.3.1 and it's included in the RPM. Removing quota works as described on the rmquota webpage. Simon Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Removing quotas in 2.3.1 broken?
Simon Matter wrote: From what I understand everything is okay here. For user.scottrus only the size of the mailbox is shown, no quota is shown because it is set to -1 in the quota db. I think the problem is that the quota file (assuming you are using quota legacy db) is not removed when setting quota to none, instead the quota is set to -1. quota -f reports for every used who has a quota file, but correctly shows no quota for it. Based n what you've said I did some further checking on both my production v2.1.16 system and my new v2.3.1 system and confirmed that they both behave exactly the same way. What exactly do you consider broken here? My php code apparently :) I think I wrongly assumed there was a problem with Cyrus because Net_Cyrus crapped all over the screen when trying to query the quota information for a mailbox that has the quota set to -1. I'll have to take a closer look at the php side of things. Thanks for restoring my sanity. -- Scott Russell [EMAIL PROTECTED] IBM Linux Technology Center System Admin Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Removing quotas in 2.3.1 broken?
Simon Matter wrote: At least with quotalegacy you can do it the dirty way by simply removing the quota file for the user. If I didn't this wouldn't I also have to run reconstruct on the mailbox? -- Scott Russell [EMAIL PROTECTED] IBM Linux Technology Center System Admin Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Removing quotas in 2.3.1 broken?
I have ported the rmquota patch to 2.3.1 and it's included in the RPM. Removing quota works as described on the rmquota webpage. Simon So probably he doesn't use it. (If i remember the first post correctly he is using cyrusadm, and not the rmquota option through imap). I'll try to have a look asap to see if the rmquota patch plays nicely with cyrus 2.3.1 or not. Thanks :) Aristotelis Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Removing quotas in 2.3.1 broken?
Aristotelis wrote: As Simon already said the quota removal works a bit differently in cyrus imap. Just a bit. I think what drives Net_IMAP_Protocol() from php pear crazy is the difference in response for GETQUOTA. If no quota db exists for a mail box then this is what cyrus says: C: GETQUOTA user.scottrus S: NO Quota root does not exist However, if the quota previously existed but was set to -1 then cyrus responds differently with: C: GETQUOTA user.scottrus S: QUOTA user.scottrus () S: OK Completed I'm not sure why cyrus responds differently depending on if the quota is -1 or if it was never set at all but it sure would be nice if it responded consistently since both situations are effectively the same. Any reason why this isn't consistent? Personally I like seeing the NO response rather than the empty quota response. NOTE: For those using php the cmdGetQuota() function from the Net_IMAP_Protocol class correctly handles the first example but not the second example. For those using the php c-client imap_get_quota() function the first example spits php errors and returns FALSE while the second example returns an empty array() without giving any php errors. The rmquota patch exists for cyrus 2.2.12. It hasn't been ported yet (officialy at least) to 2.3.1 since there wasn't much demand. If there is more demand we will port it asap to 2.3.1. (the 'auto' patches are by far more popular). I'm using this patch from Simon's 2.3.1-2 rpm and thus far it's been very helpful in testing :) -- Scott Russell [EMAIL PROTECTED] IBM Linux Technology Center System Admin Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Removing quotas in 2.3.1 broken?
Greets. I'm having problems removing quotas in Cyrus 2.3.1. I'm using Simon's 2.3.1-2 RPM. It looks like the command attempts to remove the mailbox quota but instead ends up doing something else. What I'm not sure of so lots of logs and details provided: Before starting I run quota -f to make sure everything is sane: bash-3.00$ /usr/lib/cyrus-imapd/quota -f Quota % Used Used Root 6000 user.drfickle 600 4120 user.lnxgeek 6000 user.osubeav 8007 user.scottrus Using cyradm I look at the existing quota for user.scottrus, remove it, then look at it again. [EMAIL PROTECTED] users]# cyradm --user cyrus localhost Password: localhost.localdomain lq user.scottrus STORAGE 7/80 (0.000875%) localhost.localdomain sq user.scottrus none remove quota localhost.localdomain lq user.scottrus / localhost.localdomain quit The last listquota command above returns odd results. I would expect to see blank line as with mailboxes that have been created without a quota limit set. Looking at the trace though everything looks okay: 11398677184 RLIST 1139867718* LIST (\Noselect) . 4 OK Completed (0.000 secs 0 calls) 11398677225 GETQUOTA user.scottrus 1139867722* QUOTA user.scottrus (STORAGE 7 80) 5 OK Completed 11398677276 SETQUOTA user.scottrus () 11398677276 OK Completed 11398677307 GETQUOTA user.scottrus 1139867730* QUOTA user.scottrus () 7 OK Completed However the quota command still shows some odd results for user.scottrus after attempting to remove the quota: bash-3.00$ /usr/lib/cyrus-imapd/quota -f Quota % Used Used Root 6000 user.drfickle 600 4120 user.lnxgeek 6000 user.osubeav 7 user.scottrus -- Scott Russell [EMAIL PROTECTED] IBM Linux Technology Center System Admin Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
quotas repeatedly resetting themselves
Configuration is on Mac OS X (not server) with Cyrus from Darwin cyrusIMAP-156.3We have been using this setup a little while without troubles but all of a sudden it started giving over quota warnings when we had told it to ignore quotas. Upon logging in, we discovered the quotes were set to some tiny number. When we reset them, either by editing the file or using cyradm it is only temporary. As soon as someone logs in to IMAP the quota resets back again to some tiny number: 131072 and the quota warnings restart. Anyone have ideas what it going on? Repeated attempt to reset are immediately overwritten and turning write off on the quota file is about thing to do, but generates a lot of other errors.--Thanks!--Marie Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Quotas vs. Trash revisited
On Wed, 14 Dec 2005, Rob McMahon wrote: it. It occurred to me, though, that if we made the user.xxx.Trash folder a separate quota root with the same quota as their inbox these problems would go away. Can anyone see any problems with this ? As long as you add a policy that Trash gets wiped clean every five days or somesuch, no. Otherwise, you must be prepared for users using Trash as a depot. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
