Re: Cyrus IMAPd 2.3.16 Released
Or you can use a dummy backend. It's a backend which always says « OK » when you try to write in it, and always says « not in db » when you read in it. This backend was never committed into cyrus-imapd... Here is an up-to-date version. Then add this in imapd.conf : duplicate_db: dummy Ken, Bron : do you plan to include this backend into cyrus-imapd ? It's very handy when we don't want to use a database (here we often use dummy for annotations). It looks really useful to me. Any chance this will go into upstream? I'd prefer to include it in my RPMs if I know it's also in upstream. Simon Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
RE: Quota issue
Hi all, Found another issue with quota. For some reason I get different values from the systems. At this moment Iâve set the quota to 300MB With a du âsh of the user directory I get 123MB In outlook I see 96MB With cyradm I get STORAGE 219309/30 (73.103%) In the quota file I see: 224572760 30 In a webtool I use ( web-cyradm ) I see â222543 KBytes of 30 KBytes (74,18%)â How can I make sure everything is on the same track again? Please search the list archives for more info on this. Running reconstruct and after that 'quota -f' may fix it. Regards, Simon Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Cyrus IMAPd 2.3.16 Released -- auto create patch available?
hi simon, thanks for your fantastic rpms :-) is there an autocreate patch that applies cleanly against this version? thanks m Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.16 Released -- auto create patch available?
hi simon, thanks for your fantastic rpms :-) is there an autocreate patch that applies cleanly against this version? I've updated the patch from UoA for 2.3.16. It's in my rpms. Regards, Simon Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.16 Released
On Mon, Jan 04, 2010 at 09:11:10AM +0100, Simon Matter wrote: Or you can use a dummy backend. It's a backend which always says « OK » when you try to write in it, and always says « not in db » when you read in it. This backend was never committed into cyrus-imapd... Here is an up-to-date version. Then add this in imapd.conf : duplicate_db: dummy Ken, Bron : do you plan to include this backend into cyrus-imapd ? It's very handy when we don't want to use a database (here we often use dummy for annotations). It looks really useful to me. Any chance this will go into upstream? I'd prefer to include it in my RPMs if I know it's also in upstream. It certainly will! There's a bit of 2.3 vs 2.4 uncertainty about where to put code. I think we've pretty much put 2.3 into maintainence mode. I'm telling everyone that we're releasing 2.4 in April on the theory that if you repeat something often enough it becomes true! Bron. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.16 Released -- auto create patch available?
On Mon, 2010-01-04 at 11:35 +0100, Simon Matter wrote: hi simon, thanks for your fantastic rpms :-) is there an autocreate patch that applies cleanly against this version? I've updated the patch from UoA for 2.3.16. It's in my rpms. Regards, Simon Hi Simon, H ... I've just installed roundcube web mail and am running it against my cyrus with no joy -- i.e., create a new user, login to that user's account, and no boxes are being created. Relevant bits from imapd.conf: autocreateinboxfolders: Sent|Drafts|Deleted Items|Junk autocreatequota: 204800 autosubscribeinboxfolders: Sent|Drafts|Deleted Items|Junk createonpost: yes I am now noticing that these are subfolders of Inbox -- does this patch work when altnamespace: yes is set? Cheers M Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.16 Released -- auto create patch available? [SOLVED]
On Mon, 2010-01-04 at 11:35 +0100, Simon Matter wrote: hi simon, thanks for your fantastic rpms :-) is there an autocreate patch that applies cleanly against this version? I've updated the patch from UoA for 2.3.16. It's in my rpms. Regards, Simon Turns out that the ldap management tool was creating just the Inbox, therefore, when logging into imap, the autocreate procedure was not firing. Thanks M Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: prevent stuck processes with large folder manipulations
Hi Paul On Sunday 03 January 2010 @ 13:35, Paul Dekkers wrote: Are you running 32 or 64-bits? We run 64-bits, and I realized that this allows a single imapd process to consume a considerable amount of memory (eg. all) instead of just 2G or so per process. (The server I'm talking about has 6G of memory and 2G swap, should be ok for ~50 active users.) But now there's nothing limiting the memory consumption by just one single user/process, I guess. We're running 32-bit, with 8G of ram and 8G of swap but around ~4k active users per backend machine. It suspect it would probably harmless to limit the cyrus processes to 2G with ulimit, I doubt any one imapd would ever legitimately need to be larger than that. We are not running any proxies, well, other than stunnel doing SSL offloading on another box. So apart from SSL the clients are directly talking to imapd. I do believe it's just Thunderbird timing-out. This box is running 2.3.13 (Simon's RPM on 64-bit RHEL 4), but I recall seeing it with olders versions before (of both Cyrus and TB). Actually; the mozilla bugtracker reference sounds very similar. It happens with large deletions too, just like with large copies, as described in this bug. But we see this with much more recent versions, Thunderbird 3 and the recent 2.3 cyrus. The behavior I observed was that TB was hitting a timeout, even though the server was responding almost immediately. TB just wasn't recognizing the response and it seemed related to the length of that response. If you still see the behavior with Thunderbird 3, and your able to reproduce the problem, you may be able to provide more info to the TB developers. They seem pretty responsive if they get good feedback, so there's a good chance it could be solved once and for all if you can give provide them some debugging info. Regarding the last suggestion in this bug; for the deletes I did consider the delayed expunge and/or delete, but that wouldn't help with the large copies. Even large copies should occur fairly quickly, unless you have disabled singleinstancestore. singleinstancestore is enabled by default, and causes cyrus to create hardlinks rather than actually creating copies. -Brian Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.16 Released
Bron Gondwana wrote: On Mon, Jan 04, 2010 at 09:11:10AM +0100, Simon Matter wrote: Or you can use a dummy backend. It's a backend which always says « OK » when you try to write in it, and always says « not in db » when you read in it. This backend was never committed into cyrus-imapd... Here is an up-to-date version. Then add this in imapd.conf : duplicate_db: dummy Ken, Bron : do you plan to include this backend into cyrus-imapd ? It's very handy when we don't want to use a database (here we often use dummy for annotations). It looks really useful to me. Any chance this will go into upstream? I'd prefer to include it in my RPMs if I know it's also in upstream. It certainly will! There's a bit of 2.3 vs 2.4 uncertainty about where to put code. I think we've pretty much put 2.3 into maintainence mode. I'm telling everyone that we're releasing 2.4 in April on the theory that if you repeat something often enough it becomes true! Do we really need a dummy backend, or should we just rewrite the code so that non-critical DBs can be specified as nil/none/null and just not make the database calls? Also, in the case of disabling annotation_db, we shouldn't be advertising ANNOTATEMORE/METADATA in the IMAP capabilities. In the case of disabling duplicate_db, we shouldn't be allowing the Sieve Vacation extension, and we can't prevent mail loops caused by Sieve Redirect. -- Kenneth Murchison Systems Programmer Project Cyrus Developer/Maintainer Carnegie Mellon University Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: prevent stuck processes with large folder manipulations
I'm wondering if because the COPY might be taking a long time before
responding, that the client thinks that the server has hung or gone
away. The attached (untested) patch might solve the problem.
Paul Dekkers wrote:
Hi,
From time to time (but mostly at the start of the year ;-)), I notice a
lot of load caused by people archiving their mail-folders. Maybe this is
mostly caused by Thunderbird going mad, but I was wondering if I could
do anything on the server-side to prevent things from going bad. Because
now I see memory (and swap) exhaustion and the side-effects of that
(Linux kernel killing processes)...
One example: someone was moving tens of thousands of messages from 2009
to a new 2009 folder. Apparently Thunderbird was stuck, maybe because
these things don't happen instantly moving this number of messages so
the server doesn't finish quickly: but Thunderbird created a lot (~100)
of sessions / imapd-processes for this user, maybe after timeouts.
(I think) Only one process was active doing the link's, it looked like
the others were mostly waiting for a write lock (fortunately), waiting
to do the same thing. (Inspected with strace.) But when the process that
hogged the CPU was killed, the next process took over, until all similar
processes were killed. And the new archive-folder now ended up with
several duplicates, taking about millions instead of tens of thousands.
(We'll have to see how to dedup that, any ideas are appreciated
otherwise I'll write something for that.)
It just happened, but it happened before. This mail-server is not that
busy, 100 users, but it happens at least a few times per year.
Any idea how to prevent things like this? Judging from the man-pages I
don't think I could do this from within cyrus, but that I would have to
prevent from linux's ulimit or so and tune that (sounds like a tough
job)... or could I actually do this with cyrus parameters?
Curious if people have similar experiences :-)
Regards,
Paul
P.S. This specific machine is running Red Hat 4 and a version of Simon's
(s)rpm.
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
--
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University
Index: append.c
===
RCS file: /afs/andrew/system/cvs/src/cyrus/imap/append.c,v
retrieving revision 1.109.2.2
diff -u -r1.109.2.2 append.c
--- append.c28 Dec 2009 21:51:28 - 1.109.2.2
+++ append.c4 Jan 2010 15:36:28 -
@@ -828,7 +828,8 @@
struct appendstate *as,
int nummsg,
struct copymsg *copymsg,
- int nolink)
+ int nolink,
+ struct protstream *pout)
{
struct mailbox *append_mailbox = as-m;
int msg;
@@ -841,6 +842,7 @@
int r, n;
int flag, userflag, emptyflag;
struct body *body = NULL;
+time_t start, now;
assert(append_mailbox-format == MAILBOX_FORMAT_NORMAL);
@@ -854,7 +856,15 @@
xmalloc(nummsg * sizeof(struct index_record));
/* Copy/link all files and cache info */
-for (msg = 0; msg nummsg; msg++) {
+for (start = time(0), msg = 0; msg nummsg; msg++) {
+ /* Send progress update to client every 30 sec */
+ if (pout (now = time(0)) start + 30) {
+ start = now;
+ prot_printf(pout, * OK copied %d of %d messages\r\n,
+ msg, nummsg);
+ prot_flush(pout);
+ }
+
zero_index(message_index[msg]);
message_index[msg].uid = append_mailbox-last_uid + 1 + as-nummsg;
if (append_mailbox-options OPT_IMAP_CONDSTORE) {
Index: append.h
===
RCS file: /afs/andrew/system/cvs/src/cyrus/imap/append.h,v
retrieving revision 1.28.2.2
diff -u -r1.28.2.2 append.h
--- append.h28 Dec 2009 21:51:28 - 1.28.2.2
+++ append.h4 Jan 2010 15:36:28 -
@@ -139,7 +139,8 @@
extern int append_copy(struct mailbox *mailbox,
struct appendstate *append_mailbox,
- int nummsg, struct copymsg *copymsg, int nolink);
+ int nummsg, struct copymsg *copymsg, int nolink,
+ struct protstream *pout);
extern int append_collectnews(struct appendstate *mailbox,
const char *group, unsigned long feeduid);
Index: index.c
===
RCS file: /afs/andrew/system/cvs/src/cyrus/imap/index.c,v
retrieving revision 1.219.2.10
diff -u -r1.219.2.10 index.c
--- index.c 28 Dec 2009 21:51:33 - 1.219.2.10
+++ index.c 4 Jan 2010 15:36:29 -
@@ -1814,7 +1814,7 @@
docopyuid = (append_mailbox.m.myrights ACL_READ);
r = append_copy(mailbox, append_mailbox,
Question about reconstruct
Hi, happy new year. My question is: When should I use -f flag on command reconstruct, and when should not I use it? Thanks in advance! -- -- Open Kairos http://www.sergiobelkin.com Watch More TV http://sebelk.blogspot.com Sergio Belkin - Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Question about reconstruct
On Mon, 2010-01-04 at 16:24 -0200, Sergio Belkin wrote: Hi, happy new year. My question is: When should I use -f flag on command reconstruct, and when should not I use it? From the Cyrus chapter of WMOGAG: quote The most important options for reconstruct are “-r” and “-k”. The “-r” option causes reconstruct to reindex the specified mailbox and all subordinate mailboxes, otherwise child mailboxes are not reindexed. The “-k” preserves the “cyrus.expunge” database for the mailbox and all corresponding expunged messages; this option should always be specified for servers operating in delayed-expunge mode200. If the “-k” option is not specified all expunged messages will be removed from the specified folder(s). When restoring mailboxes from an archive or backup the parameter “-f” should also be specified; with “-f” any subdirectories in the filesystem directory corresponding to the specified mailbox which contain a “cyrus.header” file will be added into the server's mailbox list. The “-x” option when provided in conjunction with “-f” will create new folders with default metadata, otherwise reconstruct will attempt to restore as much of the mailboxes metadata as possible; such as ACLs, seen state, and flags. /quote -- OpenGroupware developer: awill...@whitemice.org http://whitemiceconsulting.blogspot.com/ OpenGroupare Cyrus IMAPd documenation @ http://docs.opengroupware.org/Members/whitemice/wmogag/file_view Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Building cyrus sasl on solaris 10
Hi all!!, I'm trying to build a mail machine box with Postfix (cyrus saslauthd authentication), cyrus sasl (with saslauthd) and cyrus-imap. The main problem I'm finding for the moment is that when building cyrus sasl plugins... only static libraries are created for auth mechs causing postfix to say that no available auth mech is available (because it seems to link with dlopen dinamically...) when building cyrus sasl... I see some warnings like this while make : *** Warning: This library needs some functionality provided by -lc. *** I have the capability to make that library automatically link in when *** you link to this library. But I can only do this if you have a *** shared version of the library, which you do not appear to have. *** Warning: libtool could not satisfy all declared inter-library *** dependencies of module libplain. Therefore, libtool will create *** a static module, that should work as long as the dlopening *** application is linked with the -dlopen flag. (have tried too specifying --with-gnu-ld and compiling with sun stdio compilers and with sunfreeware gcc too used gmake too and make I have tried all I think...) So... seems like libplain.so won't be created... and this causes Postfix having this behaviour. Cyrus IMAP too fails authenticating although a testsaslauthd -u user -p pass -s smtp or imap or pop3 sais auth succesful so saslauthd works nice... So is there any procedure for building this on Solaris 10?? I know solaris comes with some parts of cyrus sasl library... but it's only one part and it doesn't come with binaries like saslauthd... so I'd rather to build whose cyrus sasl. Is it any documented way for building this properly?... or is there any documented way of running saslauthd with Postfix and cyrus IMAP I mean... any solaris way for making this work please?. Thanks a lot for you're help. Bye! -- Egoitz Aurrekoetxea Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Building cyrus sasl on solaris 10
On Mon, Jan 04, 2010 at 11:25:01PM +0100, Egoitz Aurrekoetxea wrote: So is there any procedure for building this on Solaris 10?? I know solaris comes with some parts of cyrus sasl library... but it's only one part and it doesn't come with binaries like saslauthd... so I'd rather to build whose cyrus sasl. Is it any documented way for building this properly?... or is there any documented way of running saslauthd with Postfix and cyrus IMAP I mean... any solaris way for making this work please?. Which version are you building? This is what I used to configure cyrus-sasl-2.1.22. It certainly does create dynamic libraries. env LDFLAGS=-R/usr/local/lib \ CC=cc \ INSTALL=/usr/ucb/install \ ./configure \ --localstatedir=/var/run \ --with-dblib=none \ --with-saslauthd=/var/run/saslauthd \ --with-ipctype=doors \ --with-openssl=/usr/sfw \ --with-mysql=/usr/local/src/mysql/mysql-4.0.18 \ --disable-checkapop \ --disable-otp \ --enable-login \ --enable-ntlm \ --enable-sql \ --disable-krb4 \ --disable-gssapi -- -Gary Mills--Unix Group--Computer and Network Services- Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Building cyrus sasl on solaris 10
Hi Gary, I'm doing the following configure : 670 export CXX=/opt/sunstudio12.1/bin/CC 671 export CC=/opt/sunstudio12.1/bin/cc 672 export CPPFLAGS=-I/usr/include 673 export LDFLAGS=-L/lib 674 ./configure --prefix=/export/binarioscompilados/correo/cyrussasl --with-saslauthd=/var/run --with-pam=/export/binarioscompilados/correo/pammysql/modulos --with-configdir=/export/binarioscompilados/correo/cyrussasl/etc --disable-anon --enable-plain --disable-cram --disable-digest --enable-login --disable-otp --disable-gssapi --without-krb4 --without-des --without-authdaemond And it creates libplain.a but not libplain.so Any known reason... I have tried too you're configure... but I continue seeing in make the warnings told before about libraries and dlopen... Any ideas please?... have tried too with --enable-static and --enable-shared... but no way Bye! thanks!!! 2010/1/5 Gary Mills mi...@cc.umanitoba.ca On Mon, Jan 04, 2010 at 11:25:01PM +0100, Egoitz Aurrekoetxea wrote: So is there any procedure for building this on Solaris 10?? I know solaris comes with some parts of cyrus sasl library... but it's only one part and it doesn't come with binaries like saslauthd... so I'd rather to build whose cyrus sasl. Is it any documented way for building this properly?... or is there any documented way of running saslauthd with Postfix and cyrus IMAP I mean... any solaris way for making this work please?. Which version are you building? This is what I used to configure cyrus-sasl-2.1.22. It certainly does create dynamic libraries. env LDFLAGS=-R/usr/local/lib \ CC=cc \ INSTALL=/usr/ucb/install \ ./configure \ --localstatedir=/var/run \ --with-dblib=none \ --with-saslauthd=/var/run/saslauthd \ --with-ipctype=doors \ --with-openssl=/usr/sfw \ --with-mysql=/usr/local/src/mysql/mysql-4.0.18 \ --disable-checkapop \ --disable-otp \ --enable-login \ --enable-ntlm \ --enable-sql \ --disable-krb4 \ --disable-gssapi -- -Gary Mills--Unix Group--Computer and Network Services- -- Egoitz Aurrekoetxea -- Egoitz Aurrekoetxea Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
