Re: quota problems on cyrus imapd 2.2.12
On Tue, 2006-04-25 at 13:21 -0500, Marlys Nelson wrote: Larry Rosenbaum wrote: Is anybody using more than 4GB of storage? Cyrus imapd 2.2.12 stores quota usage, in bytes, in an unsigned long and so can't keep track of usage over 4GB. You may need to go to v2.3.3, which uses a long long int. No, everyone's below 2GB in actual storage, most are either under 100MB (faculty) or 50MB (students) which is where their quotas were set until we had problems. Hi, Plenty of corporate users have years of e-mail including my boss. We are using cyrus-imapd-2.2.12-13 on SuSE 10.0 and are currently having a problem with the 4GB limit. How do I remove the quota on his cyrus mailbox (including sub-folders) and make the quota umlimited to work around the 4GB issue? Thanks Murray Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus Patches used at FastMail.FM
I've finally split out all the patches that we use here into individual items. I know some people were interested in the don't allow users to set the anyone ACL patch as well. http://cyrus.brong.fastmail.fm/ Ken - I'd love to work with you on getting as many as possible of these into upstream, at least the ones that are labelled as worth doing - some are so cheap-hack as to require a complete rewrite to make them worthwhile. Cool, some of the patches look really interesting and I'm considering to include one or the other into my rpm packages. For example the statuscache patch seems very nice. Just to be sure, are there any license restrictions on the patches? I'd love to see some of the patches integrated upstream, it makes my own work easier :) Simon Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus Patches used at FastMail.FM
--On May 23, 2006 8:37:45 AM +0200 Simon Matter [EMAIL PROTECTED] wrote: Cool, some of the patches look really interesting and I'm considering to include one or the other into my rpm packages. For example the statuscache patch seems very nice. Just to be sure, are there any license restrictions on the patches? I'd love to see some of the patches integrated upstream, it makes my own work easier :) Statuscache also piqued my interestDoes it give any win for POP3 clients? We've a *HUGE* number of Outlook users that have the terribly wrong idea that they just MUST poll every minute. That along with the seen state stuff would be a good thing for us. Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus Patches used at FastMail.FM
Cool, some of the patches look really interesting and I'm considering to include one or the other into my rpm packages. For example the statuscache patch seems very nice. Just to be sure, are there any license restrictions on the patches? No, no license restrictions. We'd love these to get back into cyrus itself, so happy to contribute them back to whatever licence cyrus is under. The statuscache is actually very useful, more so than Bron's comment on the page immediately suggests. Originally I thought it was our webmail client that was doing most of the status calls, so I actually put caching into our web client. It turned out that in fact lots of email clients also blast the IMAP server with status calls, so putting it in the server itself has made a noticeable difference to load on our servers. The other one that I've just finished today is the fastindex one, which I'm also excited about. I discovered that on our brand new server with only a couple of hundred users on it, cmdtimer was still showing that some status calls were taking 2 seconds for large mailboxes (for status calls that weren't cached that is) on a virtually unloaded server (load 0.05). Checking it out, it became pretty obvious that the problem was the core status loop which calls index_insequence() on every message to see if it's seen/unseen. When the seen state sequence list becomes quite large, you're doing an awful lot of repeated scans of the seen list. With a bit of slightly messy hacking (static variables + extra parameter) instead of cleaner but more major code surgery, I've added a caching mode when scanning an assumed sequence list of messages into a sequential seen list. This has reduced the status call to basically instant even on 40,000+ message folders with lots of random seen/unseen messages. I've put the latest version of the patch up now: http://cyrus.brong.fastmail.fm/patches/cyrus-fastindex-2.3.3.diff I'd love to see some of the patches integrated upstream, it makes my own work easier :) So would we, the less we have to maintain, the better, and for large installations with many IMAP users, the performance impact of the status and fastindex patches should be helpful. Rob Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Multiple problems with Cyrus 2.3.x
Hello, We've been running Cyrus for well over a year now supporting over 30,000 email accounts. We use a Murder setup, with virtualdomain support. We also use socketmap to insure the user exists during the smtp session. We were previously using 2.2.12 with very good luck. No real issues to speak up. There were some things in the 2.3 releases that I wanted, replication, the murder changes, and the socketmap changes (to check quota's). So we upgraded, after I did all the testing I could think of. Issues: 1) Sendmail goes completely out in the weeds. This seems to happen multiple times per day per frontend. Sendmail will spew Deferred: Connection reset by localhost messages in maillog (like has been mentioned in another thread). Unfortunately, I can't seem to get any useful debugging information as to what the cause is. I suspect it has something to do with lmtp as the problem was non-existant with 2.2.12 (tho earlier versions did have the problem, and I was never able to figure out the cause with them either). 2) We use Horde/Imp as webmail for our customers. This has also, in the past, been rock solid. Now with the upgrades users with large emails (a meg or larger) seem to not be able to receive their email at all through webmail (timeout). I can't seem to find the core issue here either, very frustrating. Both the server with webmail on it, and the Cyrus servers are on the same subnet, only seperated by a Cisco switch (which has been test, and is in perfect working order). #1 is a major major issue. It causes mail to be queue'd, sometimes for long periods of time, and I can't seem to find a specific cause other than it started after the upgrade. Any suggestion would be very welcome by my dwindling sanity. Thanks! Lenny -- Wisdom is to a man an infinite Treasure - Anonymous Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus Patches used at FastMail.FM
Statuscache also piqued my interestDoes it give any win for POP3 clients? We've a *HUGE* number of Outlook users that have the terribly wrong idea that they just MUST poll every minute. That along with the seen state stuff would be a good thing for us. No, it's only for IMAP clients and speeds up the STATUS IMAP command, it does nothing for POP3. I'm surprised you're having issues with lots of POP polling, generally POP is a lot less resource intensive than IMAP I always believed... Rob Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Archives unaccesible by http
Is int only me, or are this mailing list's archives at http://asg.web.cmu.edu/bb/archive.info-cyrus inaccessible to one and all? Mordur Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus Patches used at FastMail.FM
Bron Gondwana wrote: I've finally split out all the patches that we use here into individual items. I know some people were interested in the don't allow users to set the anyone ACL patch as well. http://cyrus.brong.fastmail.fm/ Ken - I'd love to work with you on getting as many as possible of these into upstream, at least the ones that are labelled as worth doing - some are so cheap-hack as to require a complete rewrite to make them worthwhile. I'm in the process of porting a couple of patches from UMich into 2.3 and then I'm going to make a release which includes a fix for an easily exploitable buffer overflow in pop3d. -- Kenneth Murchison Systems Programmer Project Cyrus Developer/Maintainer Carnegie Mellon University Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus Patches used at FastMail.FM
Ken Murchison wrote: Bron Gondwana wrote: I've finally split out all the patches that we use here into individual items. I know some people were interested in the don't allow users to set the anyone ACL patch as well. http://cyrus.brong.fastmail.fm/ Ken - I'd love to work with you on getting as many as possible of these into upstream, at least the ones that are labelled as worth doing - some are so cheap-hack as to require a complete rewrite to make them worthwhile. I'm in the process of porting a couple of patches from UMich into 2.3 and then I'm going to make a release which includes a fix for an easily exploitable buffer overflow in pop3d. Clicked too fast. After I make the release, I'll look at your stuff. -- Kenneth Murchison Systems Programmer Project Cyrus Developer/Maintainer Carnegie Mellon University Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus Patches used at FastMail.FM
I'm in the process of porting a couple of patches from UMich into 2.3 and then I'm going to make a release which includes a fix for an easily exploitable buffer overflow in pop3d. Ehm? Does this affect the 2.2.x branch as well? John -- John Madden Sr. UNIX Systems Engineer Ivy Tech Community College of Indiana [EMAIL PROTECTED] Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus Patches used at FastMail.FM
John Madden wrote: I'm in the process of porting a couple of patches from UMich into 2.3 and then I'm going to make a release which includes a fix for an easily exploitable buffer overflow in pop3d. Ehm? Does this affect the 2.2.x branch as well? No. The exploit is in a feature added in 2.3. -- Kenneth Murchison Systems Programmer Project Cyrus Developer/Maintainer Carnegie Mellon University Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Archives unaccesible by http
On Tue, May 23, 2006 at 12:30:42PM +, Mordur wrote: Is int only me, or are this mailing list's archives at http://asg.web.cmu.edu/bb/archive.info-cyrus inaccessible to one and all? It's working from here. Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Cyrus IMAPd 2.3.4 Released
I am pleased to announce the release of Cyrus IMAPd 2.3.4. This is a BETA-quality release, reflecting that it has significant numbers of new features that have not been tested on a wide-scale basis, although earlier versions of this code have been running at several sites for quite some time. This release fixes a potential buffer overflow in pop3d 'subfolders' support that exists in 2.3.0-2.3.3. Full details, see http://www.frsirt.com/english/advisories/2006/1891 This release also includes several new features, including the IMAP CONDSTORE extension, and support for BINARY APPEND. For full details, please see doc/changes.html and doc/install-upgrade.html which are included in the distribution. URLs for this release: ftp://ftp.andrew.cmu.edu/pub/cyrus/cyrus-imapd-2.3.4.tar.gz or http://ftp.andrew.cmu.edu/pub/cyrus/cyrus-imapd-2.3.4.tar.gz Questions and comments can be directed to info-cyrus@lists.andrew.cmu.edu (public list), or [EMAIL PROTECTED] -- Kenneth Murchison Systems Programmer Project Cyrus Developer/Maintainer Carnegie Mellon University Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: [Fwd: [Full-disclosure] Cyrus IMAPD pop3d remote compromise aka cyrusFUCK3d]
Sven Mueller wrote:
Since I didn't see this yet on either -info or -devel, I hereby forward
the security alert (if you want to call it that way, in this case, it's
a zero-day-exploit). I've investigated the issue a little bit, and as
far as I could see, the 2.2 line isn't affected by the problem. If it
is, however, I would really like to see a sufficient patch for it.
Here's the patch that's in 2.3.4:
https://bugzilla.andrew.cmu.edu/cgi-bin/cvsweb.cgi/src/cyrus/imap/pop3d.c.diff?r1=1.144.2.41r2=1.144.2.42f=u
Subject:
[Full-disclosure] Cyrus IMAPD pop3d remote compromise aka cyrusFUCK3d
From:
kcope [EMAIL PROTECTED]
Date:
Sun, 21 May 2006 13:52:05 +0200
To:
full-disclosure@lists.grok.org.uk
To:
full-disclosure@lists.grok.org.uk
Shouts to blackzero, alex, wY!, revoguard, bogus, wtfomg and all those
yankees
LOVE TO LISA :-)
genuine advisory by kcope/zeroday discovered by kcope!!!
kingcope[at]gmx.net
public disclosure 21. May 2006
vendor was not notified (mail quota exceeded) fuck it
let's get to business
-
Cyrus-imapd pop3d Remote Stack Based Buffer Overrun
Description
There is a trivially remotely exploitable Buffer Overrun in
Cyrus-imapd's pop3d.
The issue is not present in the default install, Cyrus-imapd has to have
the
popsubfolders set to 1 in imapd.conf.
From the manpage:
popsubfolders: 1
Allow access to subfolders of INBOX via POP3 by using userid+subfolder
syntax as the authentication/authorization id.
When popsubfolders is set one can overflow a stack buffer by sending an
overly long
USER command argument to the remote pop3d.
in cyrus-imapd-2.3.2/imap/pop3d.c
pop3d_canon_user is called every time a USER command is supplied to the
pop3 server
with ulen=0;
look specifically at the
char userbuf[MAX_MAILBOX_NAME+1], *p;
...
if (!ulen) ulen = strlen(user);
...
memcpy(userbuf, user, ulen);
userbuf[ulen] = '\0';
memcpy will overflow the userbuf buffer, if the user is above
MAX_MAILBOX_NAME+1,
no length check is done in this routine.
--- snip ---
static int popd_canon_user(sasl_conn_t *conn, void *context,
const char *user, unsigned ulen,
unsigned flags, const char *user_realm,
char *out, unsigned out_max, unsigned *out_ulen)
{
char userbuf[MAX_MAILBOX_NAME+1], *p;
size_t n;
int r;
if (!ulen) ulen = strlen(user);
if (config_getswitch(IMAPOPT_POPSUBFOLDERS)) {
/* make a working copy of the auth[z]id */
memcpy(userbuf, user, ulen);
userbuf[ulen] = '\0';
user = userbuf;
/* See if we're trying to access a subfolder */
if ((p = strchr(userbuf, '+'))) {
n = config_virtdomains ? strcspn(p, @) : strlen(p);
if (flags SASL_CU_AUTHZID) {
/* make a copy of the subfolder */
if (popd_subfolder) free(popd_subfolder);
popd_subfolder = xstrndup(p, n);
}
/* strip the subfolder from the auth[z]id */
memmove(p, p+n, strlen(p+n)+1);
ulen -= n;
}
}
r = mysasl_canon_user(conn, context, user, ulen, flags, user_realm,
out, out_max, out_ulen);
if (!r popd_subfolder flags == SASL_CU_AUTHZID) {
/* If we're only doing the authzid, put back the subfolder
in case its used in the challenge/response calculation */
n = strlen(popd_subfolder);
if (*out_ulen + n out_max) {
sasl_seterror(conn, 0, buffer overflow while canonicalizing);
r = SASL_BUFOVER;
}
else {
p = (config_virtdomains (p = strchr(out, '@'))) ?
p : out + *out_ulen;
memmove(p+n, p, strlen(p)+1);
memcpy(p, popd_subfolder, n);
*out_ulen += n;
}
}
return r;
}
--- snip ---
Attached is a working zeroday exploit for the mentioned vulnerability.
It brute forces the stack offset to spawn a uid/gid cyrus shell.
Maybe it needs a bit of tweaking to exploit cyrus servers in the wild,
since it was only tested locally on a debian install.
No further attempts to gain uid/gid root where done... maybe setreuid in
shellcode?
/* zeroday warez
* !!! PRIVATE - DONT DISTRIBUTE - PRIVATE !!!
*
* cyruspop3d.c - cyrus pop3d remote exploit by kcope
* tested on cyrus-imapd-2.3.2,linux
*
* bug found 23 Apr 2006 by kcope
*
*
* imapd/pop3d.c line 1830 :
* char userbuf[MAX_MAILBOX_NAME+1], *p;
* ...
* if (!ulen) ulen = strlen(user);
* if (config_getswitch(IMAPOPT_POPSUBFOLDERS)) {
*memcpy(userbuf, user, ulen);
*userbuf[ulen] = '\0';
* ...
* popsubfolders has to be enabled
*
* thnx to blackzero revoguard wY! qobaiashi bogus alex
* Love to Lisa :-)
*
* !!! PRIVATE - DONT DISTRIBUTE - PRIVATE !!!
*/
#include stdio.h
#include stdlib.h
#include string.h
#include sys/types.h
Re: Cyrus IMAPd 2.3.4 Released
Hi, On Tue, 23 May 2006 10:44:15 -0400 Ken Murchison [EMAIL PROTECTED] said: murch I am pleased to announce the release of Cyrus IMAPd 2.3.4. This is a murch BETA-quality release, reflecting that it has significant numbers of new murch features that have not been tested on a wide-scale basis, although murch earlier versions of this code have been running at several sites for murch quite some time. Are there any backward compatibility issue against 2.3.3? The lmtpd cannot accept any new message with following message: May 24 01:03:55 ameno lmtpunix[37779]: accepted connection May 24 01:03:55 ameno lmtpunix[37779]: lmtp connection preauth'd as postman May 24 01:03:55 ameno lmtpunix[37779]: verify_user(user.ume) failed: Mailbox has an invalid format And, my MUA complains to cannot find %INBOX. I'm not sure they are related to this problem, but I have following settings in my imapd.conf: unixhierarchysep: yes altnamespace: yes duplicatesuppression: no flushseenstate: yes Sincerely, -- Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan [EMAIL PROTECTED] [EMAIL PROTECTED],jp.}FreeBSD.org http://www.imasy.org/~ume/ Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.4 Released
Hajimu UMEMOTO wrote: Hi, On Tue, 23 May 2006 10:44:15 -0400 Ken Murchison [EMAIL PROTECTED] said: murch I am pleased to announce the release of Cyrus IMAPd 2.3.4. This is a murch BETA-quality release, reflecting that it has significant numbers of new murch features that have not been tested on a wide-scale basis, although murch earlier versions of this code have been running at several sites for murch quite some time. Are there any backward compatibility issue against 2.3.3? The lmtpd cannot accept any new message with following message: There *shouldn't* be. The cyrus.index format has changed, bu the mailboxes should be upgraded on the fly, just like with previous changes. May 24 01:03:55 ameno lmtpunix[37779]: accepted connection May 24 01:03:55 ameno lmtpunix[37779]: lmtp connection preauth'd as postman May 24 01:03:55 ameno lmtpunix[37779]: verify_user(user.ume) failed: Mailbox has an invalid format And, my MUA complains to cannot find %INBOX. If you reconstruct the mailbox, do the problems go away? I'm not sure they are related to this problem, but I have following settings in my imapd.conf: unixhierarchysep: yes altnamespace: yes duplicatesuppression: no flushseenstate: yes Sincerely, -- Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan [EMAIL PROTECTED] [EMAIL PROTECTED],jp.}FreeBSD.org http://www.imasy.org/~ume/ -- Kenneth Murchison Systems Programmer Project Cyrus Developer/Maintainer Carnegie Mellon University Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus Patches used at FastMail.FM
Robert Mueller wrote: Cool, some of the patches look really interesting and I'm considering to include one or the other into my rpm packages. For example the statuscache patch seems very nice. Just to be sure, are there any license restrictions on the patches? No, no license restrictions. We'd love these to get back into cyrus itself, so happy to contribute them back to whatever licence cyrus is under. The statuscache is actually very useful, more so than Bron's comment on the page immediately suggests. Originally I thought it was our webmail client that was doing most of the status calls, so I actually put caching into our web client. It turned out that in fact lots of email clients also blast the IMAP server with status calls, so putting it in the server itself has made a noticeable difference to load on our servers. I know we discussed this in the past, but I can't seem to find the thread. What part of the existing STATUS code causes the bottleneck? Is it STATUS_RECENT and STATUS_UNSEEN? -- Kenneth Murchison Systems Programmer Project Cyrus Developer/Maintainer Carnegie Mellon University Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.4 Released
Hi, On Tue, 23 May 2006 12:38:12 -0400 Ken Murchison [EMAIL PROTECTED] said: May 24 01:03:55 ameno lmtpunix[37779]: accepted connection May 24 01:03:55 ameno lmtpunix[37779]: lmtp connection preauth'd as postman May 24 01:03:55 ameno lmtpunix[37779]: verify_user(user.ume) failed: Mailbox has an invalid format And, my MUA complains to cannot find %INBOX. murch If you reconstruct the mailbox, do the problems go away? Yes, it fixed the problem. Thank you. However, when I update an index from my MUA, new messages don't appear in the index at all. I'm sure there are new messages in my mailbox, though. I downgraded to 2.3.3, then the new messages appeared in the index of my MUA. Any idea? Sincerely, -- Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan [EMAIL PROTECTED] [EMAIL PROTECTED],jp.}FreeBSD.org http://www.imasy.org/~ume/ Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus Patches used at FastMail.FM
--On May 23, 2006 10:34:20 PM +1000 Robert Mueller [EMAIL PROTECTED] wrote: Statuscache also piqued my interestDoes it give any win for POP3 clients? We've a *HUGE* number of Outlook users that have the terribly wrong idea that they just MUST poll every minute. That along with the seen state stuff would be a good thing for us. No, it's only for IMAP clients and speeds up the STATUS IMAP command, it does nothing for POP3. I'm surprised you're having issues with lots of POP polling, generally POP is a lot less resource intensive than IMAP I always believed... Well being as the POP3d runs the mailbox list every time a client connects. :) It certainly would also help because a lot of our accesses are via webmail as well. Rob -- Genius might be described as a supreme capacity for getting its possessors into trouble of all kinds. -- Samuel Butler Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Imap slow opening folder
Hi all! I have a setup where Cyrus-Imap 2.1.15, Cyrus-Sasl 2.1.15, Sendmail 8.13 is installed on an HPUX Itanium server. I use with success Saslauthd that authenticate users over an openldap linux RH3 server using pam modules. All is fine user can login and receive emails from Sendmail with no delay but I have only one issue, when users open their imap folders for the first time (using webmail or thunderbird) the response from the server is affcted by a delay of over 15 sec also with no messages in the folders. After this time is elapsed all is extremely fast! Can you help me? Excuse for my english and thanks. Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.4 Released
Hajimu UMEMOTO wrote: Hi, On Tue, 23 May 2006 12:38:12 -0400 Ken Murchison [EMAIL PROTECTED] said: May 24 01:03:55 ameno lmtpunix[37779]: accepted connection May 24 01:03:55 ameno lmtpunix[37779]: lmtp connection preauth'd as postman May 24 01:03:55 ameno lmtpunix[37779]: verify_user(user.ume) failed: Mailbox has an invalid format And, my MUA complains to cannot find %INBOX. murch If you reconstruct the mailbox, do the problems go away? Yes, it fixed the problem. Thank you. However, when I update an index from my MUA, new messages don't appear in the index at all. I'm sure there are new messages in my mailbox, though. I downgraded to 2.3.3, then the new messages appeared in the index of my MUA. Any idea? Not off the top of my head. I'll look into this tomorrow. -- Kenneth Murchison Systems Programmer Project Cyrus Developer/Maintainer Carnegie Mellon University Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Replication specifics
So I got into a big argument with the people in my department about how replication works and I'm seeking some guidance from the community: (1)The worst fear of any prof here at UMASS is the potential of losing a single email. So my question is this: If we set up replication, and we have to failover to the replica, is there any way to get back email that may not have been replicated -- ones that currently only exists on the defunct master? If the replica updates every 10 seconds, then we have the potential to lose 10 seconds of email. Or worse case, the sync_client dies and we lose 30 minutes or more of emails before we failover! Do other folks out there plan for this potential for lost emails or do you just failover and if a few messages get lost, you don't worry about it? (2)Also, is there a master sync transaction log file somewhere that specifies what is being done? In other words, if we failed over, could we find a transaction log that would tell us what was not committed and then manually run through it to make the updates? I found the log files in /var/lib/imap/sync, but these are very uninformative: for example: SEEN davidk user.davidk SEEN davidk user.davidk SEEN davidk user.davidk it would be nice to see SEEN update message READ 12020 for user.davidk.INBOX, but I don't know if this detailed information is somewhere on the system or just resides in memory. (3) My final question is this: If we do a manual sync_client update, is the update a full copy or is it a differential copy? So I want to know if we run a manual sync_client if it is going to overwrite the entire replica's mailstore or just search and find what is different and just update those portions. Thank you kindly David -- David Korpiewski Phone: 413-545-4319 Software Specialist IFax: 413-577-2285 Department of Computer Science ICQ: 7565766 University of Massachusetts Amherst Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Replication specifics
On May 23, 2006, at 4:48 PM, David Korpiewski wrote: So I got into a big argument with the people in my department about how replication works and I'm seeking some guidance from the community: (1)The worst fear of any prof here at UMASS is the potential of losing a single email. So my question is this: If we set up replication, and we have to failover to the replica, is there any way to get back email that may not have been replicated -- ones that currently only exists on the defunct master? If the replica updates every 10 seconds, then we have the potential to lose 10 seconds of email. Or worse case, the sync_client dies and we lose 30 minutes or more of emails before we failover! Once we have the primary/master backend machine working again after a failover (assuming its RAID is still intact) we do a find for any messages that have timestamps just prior to the the machine failing. We then compare this list to the messages on the replica. Since we have delayed expunge on, we can still determine if a specific message was replicated even if the user deleted it. We also monitor the sync_client process and someone gets alerted if it goes away. Of course some messages can be lost. But the same is true for any of your smtp machines. If one suffers a catastrophic failure then any messages queued on the machine would be lost. Do other folks out there plan for this potential for lost emails or do you just failover and if a few messages get lost, you don't worry about it? (2)Also, is there a master sync transaction log file somewhere that specifies what is being done? In other words, if we failed over, could we find a transaction log that would tell us what was not committed and then manually run through it to make the updates? I found the log files in /var/lib/imap/sync, but these are very uninformative: for example: SEEN davidk user.davidk SEEN davidk user.davidk SEEN davidk user.davidk it would be nice to see SEEN update message READ 12020 for user.davidk.INBOX, but I don't know if this detailed information is somewhere on the system or just resides in memory. We look there as well (and back it up prior ). Then we just look in the users' folders for the timestamps on messages. (3) My final question is this: If we do a manual sync_client update, is the update a full copy or is it a differential copy? So I want to know if we run a manual sync_client if it is going to overwrite the entire replica's mailstore or just search and find what is different and just update those portions. I believe it does a diff (I haven't looked at the code) -Patrick Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Imap slow opening folder
I would guess the slowdown is somewhere in your authentication process. Are LDAP lookups fast? What happens when you run this: # testsaslauthd -u username -p password ? Davide Pasquale wrote: Hi all! I have a setup where Cyrus-Imap 2.1.15, Cyrus-Sasl 2.1.15, Sendmail 8.13 is installed on an HPUX Itanium server. I use with success Saslauthd that authenticate users over an openldap linux RH3 server using pam modules. All is fine user can login and receive emails from Sendmail with no delay but I have only one issue, when users open their imap folders for the first time (using webmail or thunderbird) the response from the server is affcted by a delay of over 15 sec also with no messages in the folders. After this time is elapsed all is extremely fast! Can you help me? Excuse for my english and thanks. Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- Jules Agee System Administrator Pacific Coast Feather Co. [EMAIL PROTECTED] x284 Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus Patches used at FastMail.FM
I know we discussed this in the past, but I can't seem to find the thread. What part of the existing STATUS code causes the bottleneck? Is it STATUS_RECENT and STATUS_UNSEEN? It's a combination of both. The main things are: 1. Status UNSEEN and RECENT both have to loop over the cyrus.index file. With users with many folders and large message counts that don't change that often, it doesn't stop their IMAP clients issuing a STATUS call on every folder every 5 minutes. On a busy server, even with lots of memory the system disk cache will be completely flushed over a 5 minute period, so these STATUS calls cause every cyrus.index file to be re-read into memory. Given most folders don't change that often, with the cache these STATUS calls are serviced without having to read the cyrus.index file at all. 2. Status UNSEEN was particularly slow with large seen sequences. The fastindex patch mostly fixes that is it's not as much of an issue any more. Rob Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Removing quota (was: Re: quota problems on cyrus imapd 2.2.12)
Using cyradm, use sq mailbox none. Hope that solves your problem, Baltsar Thanks for the reply. The above worked on user.mailbox but they have lots of sub-folders. Is there any way to do this without scripting? Murray Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Implementing IMAP advice for first timer
FreeBSD-4.11 Well I want to get IMAP running on my FreeBSD box and would like to have a safe, non service-interrupting strategy to implementing it. I am leaning toward installing cyrus imapd. I have some questions about how to get things working. 1) Can somebody please recommend a good FAQ about how-to get IMAP running my FreeBSD machine? 2) Are there things I should be aware of before I start the process? 3) I dont completely understand how IMAP works is there a good tutorial about this subject? 4) I dont completely understand how local mail delivery will change is there a good tutorial about this subject? Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
