Re: CPU usage issue with Append command (cyrus 3.0.6)
For starters: strace one of the processes & see if it's making any system calls at all. pstack a process multiple times to get PC samples. gcore a process to get a core dump & take a look at the core dump with gdb. If you know that a specific account is likely to have this issue, enable IMAP protocol logging for the account. This may tell you something. Ben On 05/08/2018 02:04 PM, Andrew Nichols via Info-cyrus wrote: Hello, We've recently upgrade to cyrus 3.0.6 and we've been experiencing issue where imapd will get stuck in an Append and take 100% CPU time. The client doesn't see anything weird. I've tried leaving the processes to see if they resolve, but they never did. Doing a kill -TERM twice will kill the process. I did not notice any issues with cyrus 3.0.5, but we've been trying to get Xapian indexing going, so we moved to 3.0.6. Does anyone have any information on where I can look to rectify this issue? Here's an example of the ps output: 87556 cyrus 1 102 0 163M 24192K CPU7 7 1:56 96.71% imapd: imap: hostname> [] user.. Append (imapd) 89680 cyrus 1 102 0 163M 24280K CPU4 4 3:10 96.22% imapd: imap: hostname> [] user.. Append (imapd) 43932 cyrus 1 102 0 161M 23416K CPU5 5 1:41 96.17% imapd: imap: hostname> [] user.. Append (imapd) 81590 cyrus 1 102 0 163M 24976K CPU6 6 3:14 94.76% imapd: imap: hostname> [] user.. Append (imapd) -- Ben Carter University of Pittsburgh/CSSD Network Operations Center b...@pitt.edu 412-624-6470 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Refuse IMAP without encryption
Try adding: allowplaintext: 0 to imapd.conf Ben On 04/23/2013 09:09 AM, Paul van der Vlis wrote: Hello, Is it possible to refuse IMAP-access without encryption like TLS or SSL? I think this would be a good idea for security. And I would like to make an exception for localhost for the webmail. The webmail (Sogo) can do TLS or SSL, but normally I don't do that for localhost. I am using Cyrus 2.4.16 from Debian 7 (Wheezy). With regards, Paul van der Vlis. -- Ben Carter University of Pittsburgh/CSSD b...@pitt.edu 412-624-6470 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Another 2.4 upgrade horror story
Can you tell us more about your storage configuration? Ben -- Ben Carter University of Pittsburgh/CSSD b...@pitt.edu 412-624-6470 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Detect if nntp support is enabled
andreas.moro...@sb-brixen.it wrote: Hello, is there a way to detect if nntp support is enabled in cyrus ? Thanks Andreas Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html telnet to port 119? Ben -- Ben Carter University of Pittsburgh/CSSD b...@pitt.edu 412-624-6470 Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Strange LMTP crash
Michael Bacon wrote:
Hi, all,
I'm working through a bizarre segfault from lmtpd that occurs following a
rcpt to: command. The best I can describe what's going on is that somehow
the NULL value stored in the authstate pointer is getting changed to
0x1010101 when passed to the verify_user function. Here's a relevant GDB
snippet:
#5 0x00025950 in process_recipient (addr=0x172fbf , namespace=0x162610,
ignorequota=0, verify_user=0x21310 verify_user, msg=0x179c08)
at /opt/local/src/cyrus-imapd-2.3.14/imap/lmtpengine.c:901
901 in /opt/local/src/cyrus-imapd-2.3.14/imap/lmtpengine.c
(gdb) print msg-authstate
$7 = (struct auth_state *) 0x0
(gdb) print *msg
$8 = {data = 0x0, f = 0x0, id = 0x0, size = 0,
return_path = 0x172f58 mich...@snowplow.org, rcpt = 0x172188,
rcpt_num = 0, authuser = 0x0, authstate = 0x0, rock = 0x0,
hdrcache = 0x17dda0}
(gdb) down
#4 0x0002163c in verify_user (user=0x16f950 baconm, domain=0x0,
mailbox=0x0, quotacheck=0, authstate=0x1010101)
at /opt/local/src/cyrus-imapd-2.3.14/imap/lmtpd.c:1037
So process_recipient is calling verify_user with the correct value from
msg-authstate (0x0, although this is odd, since by this point I should be
authenticated, but whatever...). Once the process enters verify_user,
however, gdb shows that value as 0x1010101.
At some point down the line, the code checks to see if there's a value in
the pointer, and because there is, it proceeds to try to dereference
0x1010101 (in strcmp), resulting in the SEGV.
I'm still looking, but has anyone seen anything like this before?
-Michael
Backtrace below:
#0 0xfec31b60 in strcmp () from /lib/libc.so.1
#1 0x000a1a94 in mymemberof (auth_state=0x1010101,
identifier=0x172ff8 baconm)
at /opt/local/src/cyrus-imapd-2.3.14/lib/auth_unix.c:84
#2 0x000a18d0 in auth_memberof (auth_state=0x1010101,
identifier=0x172ff8 baconm)
at /opt/local/src/cyrus-imapd-2.3.14/lib/auth.c:94
#3 0x000a1110 in cyrus_acl_myrights (auth_state=0x1010101,
acl=0x172ff8 baconm)
at /opt/local/src/cyrus-imapd-2.3.14/lib/acl_afs.c:91
#4 0x0002163c in verify_user (user=0x16f950 baconm, domain=0x0,
mailbox=0x0, quotacheck=0, authstate=0x1010101)
at /opt/local/src/cyrus-imapd-2.3.14/imap/lmtpd.c:1037
#5 0x00025950 in process_recipient (addr=0x172fbf , namespace=0x162610,
ignorequota=0, verify_user=0x21310 verify_user, msg=0x179c08)
at /opt/local/src/cyrus-imapd-2.3.14/imap/lmtpengine.c:901
#6 0x0002801c in lmtpmode (func=0x158024, pin=0x179a38, pout=0x179ab0,
fd=0)
at /opt/local/src/cyrus-imapd-2.3.14/imap/lmtpengine.c:1534
#7 0x0001ec3c in service_main (argc=1, argv=0x16f618, envp=0xffbffcd4)
at /opt/local/src/cyrus-imapd-2.3.14/imap/lmtpd.c:299
#8 0x0001e610 in main (argc=1, argv=0xffbffccc, envp=0xffbffcd4)
at /opt/local/src/cyrus-imapd-2.3.14/master/service.c:540
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
I already found this. The quota that is passed to verify_user() by
process_recipient() has to be a quota_t, not an int. A patch was
already submitted.
Ben
--
Ben Carter
University of Pittsburgh/CSSD
b...@pitt.edu
412-624-6470
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Upgrade and Migration
Carson Gaspar wrote: Ben Carter wrote: Carson Gaspar wrote: Ben Carter wrote: You have to do the _exact_ same thing with imapsync, unless you want to lose email. As has already been pointed out, you are incorrect. The order is: [Pre-create inboxes with large quotas on new server] 1. Shut off mail delivery to old server 2. Shut off imapd on old server. 3. Bring up new server with imapd running 4. Start mail delivery to new server. 5. Start imapd on old server with a new IP address or bound to a nonstandard port so MUAs will not get to it. 6. Migrate data using imapsync. And your service can be down literally for only a few minutes if you plan correctly. You have chosen to have users see missing data. You have made a tradeoff You can't see missing data ;) between correctness and length of downtime by bringing up your new server in a client visible way before you sync your data. I (foolishly, it seems) assumed that you wouldn't want your users to log in to empty mailboxes, and would leave your new server non-user-visible until your sync completed. Yes, that's the trade-off, obviously. We forewarned our users a week or 2 ahead of time though, and let's face it: a lot of retained e-mail just spins around out there, never to be accessed again. This is something done on the weekend and/or at night too, remember. Also, in addition to being forewarned, each user got a start-of-migration message and an end-of-migration message so they knew exactly what was happening and they knew when their migration was complete. So, migrating like this, the users can send and receive mail with almost no interruption, and ours seemed to be quite happy with this. We didn't hear a peep, and we have a lot of users. I think that it's much more important to them to be able to send and receive e-mail without interruption than to have their old mail present immediately. And, their old mail shows up just as it was in a relatively short period of time. Ben -- Ben Carter University of Pittsburgh/CSSD b...@pitt.edu 412-624-6470 Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Upgrade and Migration
Carson Gaspar wrote: Ben Carter wrote: If you use rsync, you have to stop everything until that finishes, possibly reconstruct all mailboxes, maybe fix some other things before giving people their mail functionality back and allowing mail delivery to resume. That's just silly. If you're going to use rsync to migrate data, you do at least one rsync while the source data is live. More than one if the initial sync takes a long time. Then you go offline, do a final sync (which should be very fast), and bring the new data store online. The longer you wait to upgrade, the less silly it gets. In the case of a move to completely new servers, doing a migration using the protocol can be much more appealing. Also, we did what you suggest with rsync on our previous upgrade and we did an rsync every night leading up to the upgrade night, so the last rsync copied only a day's worth of changes. The last rsync took hours even though we ran multiple rsyncs concurrently. I think it may have taken as long as 6-8 hours, but I don't remember the exact timing. You have to do the _exact_ same thing with imapsync, unless you want to lose email. As has already been pointed out, you are incorrect. The order is: [Pre-create inboxes with large quotas on new server] 1. Shut off mail delivery to old server 2. Shut off imapd on old server. 3. Bring up new server with imapd running 4. Start mail delivery to new server. 5. Start imapd on old server with a new IP address or bound to a nonstandard port so MUAs will not get to it. 6. Migrate data using imapsync. And your service can be down literally for only a few minutes if you plan correctly. Also, the ACL format in the mailboxes file might be different between these 2 Cyrus versions. Might be, but I don't think it is. If you use the protocol to move the data, you don't have to worry about any data structure differences etc. You also can re-arrange your partitions and so on. Plus it re-calculates all quota usage as imapsync APPENDs the messages during the migration. All true, except to the best of my knowledge none of this (except repartitioning, which the OP didn't mention) matters for cyrus imapd - it will Just Work(tm) on your old data store. The only exceptions are database format changes (if you use bdb and you've revved the library version, for example), and sieve compiled bytecode. Yes, but that's the point: you can fix all those things that you always wanted to fix. Spin out the mailboxes evenly again across partitions, change DB formats, use fulldirhash, go to fewer, larger partitions, leave unused mailboxes behind, etc. Not to mention that the years of RENAME operations you did to move mailboxes left garbage behind that rsync would blindly copy over. And why do you care about quota re-calculation? The existing quota data should be correct. In our case, the problem was that the old code used 32-bit integers to track quota/usage so we had to have a cron job that zeroed usage on the old server for large mailboxes every once in a while. Ben -- Ben Carter University of Pittsburgh/CSSD b...@pitt.edu 412-624-6470 Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Upgrade and Migration
Garry wrote: Simon Matter wrote: I think an easier way is to use the 'proxyservers' option to declare a proxy user allowed to access all mailboxes. This user can then be used by imapsync. I'm quite sure the archives hold detailed information on this as I don't remember all the details. Cool! Works, at least accessing the user mail directories ... guess I will still have to figure out the best way of transferring all the access rights to the new server, but this should at least get the mails copied more easily ... -garry Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html Yes, the proxy login worked for us. If by access rights you mean IMAP ACLs, the imapsync option for this is --syncacls. Ben -- Ben Carter University of Pittsburgh/CSSD b...@pitt.edu 412-624-6470 Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Upgrade and Migration
Peter Clark wrote:
Hello,
I have an old cyrus install (2.1.16) with 30GB of data that I need to
migrate to a new server. My challenge is that something on that server
is screwed up and I have no access to PERL on that machine anymore (the
OS is past EOL, so it is long past time to move anyways). I have a new
box (2.3.14) and a bunch of trepidation about screwing this move up. I
have looked at the Cyrus Upgrade doc and seemingly this is a rather
straight forward operation.
Does this make sense?
1.rsync (pulling from the old server to the new):
config directory (/var/imap)
partition-default (/var/spool/imap)
2. /usr/cyrus/bin/cvt_cyrusdb /var/imap/mailboxes.db berkeley
/var/imap/mailboxes.db.new skiplist
3. mv /var/imap/mailboxes.db.new /var/imap/mailboxes.db
4. find /var/imap/user -name \*.seen -exec /usr/cyrus/bin/cvt_cyrusdb
\{\} flat \{\}.new skiplist \; -exec mv \{\}.new \{\} \;
5. start imapd
Is that it? I just want to make sure I am not missing anything.
Thank you,
Peter
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Peter,
You might want to use imapsync to do this. It uses the IMAP protocol,
copies flags, can copy ACLs etc. What I did was to create all the
INBOXes on the new server, set up large quotas on the new server,
point mail delivery to the new server and then use imapsync to copy
the folders and messages. People had full mail functionality right
away and their folders/messages were backfilled.
At the end, I set quotas back to what I really wanted them to be.
If you use rsync, you have to stop everything until that finishes,
possibly reconstruct all mailboxes, maybe fix some other things before
giving people their mail functionality back and allowing mail delivery
to resume.
Also, the ACL format in the mailboxes file might be different between
these 2 Cyrus versions.
If you use the protocol to move the data, you don't have to worry
about any data structure differences etc. You also can re-arrange
your partitions and so on. Plus it re-calculates all quota usage as
imapsync APPENDs the messages during the migration.
You'll have to enable proxy logins on both IMAP servers to do this
administratively with imapsync.
I copied 1.2TB, ~65,000 users in ~28 hours by using up to 128
concurrent imapsync processes at a time this way.
Ben
--
Ben Carter
University of Pittsburgh/CSSD
b...@pitt.edu
412-624-6470
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: backup without stopping the imap server?
Markus Heller wrote: Dear list, I would like to ask once again as I have not received a positive answer yet. Does the cyrus imap server meanwhile offer the feature of a seemless / checkpointed backup that does not require to stop the service? Thanks in advance, If you're on Solaris, take a look at fssnap (1m). No need to stop anything and you'll get an exact atomic point in time copy of the filesystem. If you're using VxFS, look at Snapshot Filesystems - same thing essentially. Ben -- Ben Carter University of Pittsburgh/CSSD [EMAIL PROTECTED] 412-624-6470 --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: high-availability again
zorg wrote: Hi c ould you give me just some more explaination of what is the stage./ files used during LMTP delivery have unique filenames so if i underdstand what you saying. if the stage./ files used during LMTP delivery is the same for all the node of the cluster share the same SAN then there won't be any problem thank In imap/append.c (at least in our Cyrus version) there is a function called append_newstage, and lmtpd uses this routine for mail delivery. The name of the temporary file being created in this code is of the form pid-timestampseconds which of course is not guaranteed to be unique across the cluster, so we just changed this code to create a filename of the form clusternode-pid-timestampseconds. (If you truss/strace lmtpd during message delivery, you'll understand this right away) Probably what should also happen in the std. code if some cluster support is officially added is that master should use some exclusive locking mechanism when it starts, to guarantee that a sysadmin's typo in imapd.conf won't allow cluster nodes to share a node ID, if the technique of configuring a cluster node ID in imapd.conf is used. And probably also, a node ID should always be required (which our code does). When we get a chance, we're going to talk to Derrick about getting some cluster support into the std. code. Ben -- Ben Carter University of Pittsburgh/CSSD [EMAIL PROTECTED] 412-624-6470 --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: high-availability again
Dave McMurtrie wrote: Amos wrote: What sort of changes did you have to make? We just had to change map_refresh() to call mmap() with MAP_PRIVATE instead of MAP_SHARED. Since mmap() is being called with PROT_READ anyway, this doesn't affect the operation of the application since the mapped region can never be updated. Veritas CFS was not very efficient about maintaining cache coherency across all cluster nodes when we were using MAP_SHARED. Everything worked, but under heavy load it became extremely slow. Actually, the important code change for any active/active cluster configuration is to make sure the stage./ files used during LMTP delivery have unique filenames across the cluster. There are some other setup differences related to this same issue such as symlinking /var/imap/proc, /var/imap/socket, and if you care /var/imap/log to local filesystem space on each cluster node. You could make these filenames unique across the cluster with code changes if you want to make the code changes for these also. We added a clusternode parameter to imapd.conf to accomplish this for the LMTP stage./ files. Otherwise, it just worked. Ben -- Ben Carter University of Pittsburgh/CSSD [EMAIL PROTECTED] 412-624-6470 --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Question about lmtp.lock
info-cyrus -- Amos Gouaux wrote: David G Mcmurtrie wrote: On Mon, 13 Dec 2004, Michael Loftis wrote: There are a number of different lockfiles that cyrus uses during it's lifecycle. Though one thing you need to know is that it is not designed to run like you're attempting to run it. Each cyrus instance is meant to be standalone, and running them all from a shared disk could very likely cause mailbox corruption, as well as skiplist corruption for the seen/subs entries and such. The only part of cyrus that may be multi-node from same storage safe is the BerkeleyDB stuff used for the mailboxes and deliver DBs. The cluster filesystem honors POSIX locking semantics across all nodes in the cluster. Take a look at MURDER, that's what you'll want if you want to cluster Cyrus. MURDER+heartbeat with a shared storage medium provides for failover (heartbeat is a separate, generic failover package). We chose a clustered environment over murder because we wanted scalability and redundancy, not just scalability. You're using the Cyrus code from CVS (3.0 branch?), right? Based on what I've picked up in the past, you'll need that for such a clustered environment. I know Ken had done some work on such a layout, but I don't know what the present state is. (I think in that situation there were some issues with Sun's QFS, but not sure.) Amos We're expecting very few issues here - if any. As long as the locking works across the cluster (and it is supposed to of course), the only issue we expect is to have to add a VxFS-specific ioctl call to the Cyrus code to extend the flat mailboxes file in large chunks - maybe just one extension operation. That's because in Veritas' CFS, a single node masters a given filesystem for metadata updates. Also, I can't recommend QFS. In my benchmarks it was a lot slower than Veritas' CFS. Ben -- Ben Carter University of Pittsburgh/CSSD [EMAIL PROTECTED] 412-624-6470 --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
deliver.db and related issues
Cyrus: v2.1.15 OS: Solaris 8 Berkeley DB 4.1.25 I believe I read all of the recent threads on deliver.db issues in the archives for this mailing list, but I still have some questions: First of all we have duplicate delivery suppression turned off. However, deliver.db still seems to be maintained. I saw a post about it being needed for sieve too but we don't plan to use sieve either. My question is, can I totally disable the use of deliver.db without making code changes? (For now, we changed deliver.db to skiplist since BDB format was causing hung and 'signaled to death by 10/11' lmtpd processes) Also, can the periodic execution of ctl_cyrusdb -c be safely disabled if one used a cron entry to make copies of the mailboxes file etc? Finally, since we have no Berkely DB files, what are all the __db... files that get created in the 'db' subdirectory and why is one of them 2MB in size? Our test mailboxes file is only 72 bytes... Thanks, Ben Ben Carter University of Pittsburgh [EMAIL PROTECTED] 412-624-6470
Re: deliver.db and related issues
Rob Siemborski wrote: On Wed, 15 Oct 2003, Ben Carter wrote: First of all we have duplicate delivery suppression turned off. However, deliver.db still seems to be maintained. I saw a post about it being needed for sieve too but we don't plan to use sieve either. My question is, can I totally disable the use of deliver.db without making code changes? No. OK. (For now, we changed deliver.db to skiplist since BDB format was causing hung and 'signaled to death by 10/11' lmtpd processes) Also, can the periodic execution of ctl_cyrusdb -c be safely disabled if one used a cron entry to make copies of the mailboxes file etc? You probably still want to checkpoint, since otherwise recovery will take much longer. Note that this is more or less a noop for skiplist. We're thinking of having a cron job that locks the mailboxes file (flat file) and just copies it to a different disk occasionally. So, I'm assuming in this case that I wouldn't need recovery using ctl_cyrusdb at all, since we don't care about the deliver.db file's recovery. Am I correct? I'm not trying to re-invent the wheel, but just trying to simplify the Cyrus part of our server's configuration. I don't like ctl_cyrusdb calling BDB routines at all (especially since we have no BDB files) because if it gets hung or spews error messages to syslog, both of which it's done to us, we'd rather just avoid it. If there is a way to get ctl_cyrusdb to just backup the flat mailboxes file and do its noop for the skiplist deliver.db that would be fine with us too. Finally, since we have no Berkely DB files, what are all the __db... files that get created in the 'db' subdirectory and why is one of them 2MB in size? Our test mailboxes file is only 72 bytes... Those are Berkeley DB log files. Did you have a berkeley db setup at one time? We did, but I wiped out db/*, db.backup1/* and db.backup2/* and these files reappeared - I assume they were created by ctl_cyrusdb. Thanks, Ben Ben Carter University of Pittsburgh [EMAIL PROTECTED] 412-624-6470
Highly available and scalable Cyrus configurations
We currently have a single IMAP server and we are considering two Solaris 8/9 HA Cyrus configurations: #1. IMAP Aggregator (murder) #2. Cyrus on a Veritas cluster (common file system with parallel service groups) After looking at the murder design, it seems to me that it's a lot more complicated than having a Veritas cluster and simply running imapd and an MTA on each cluster member, because with the murder configuration, you probably have to end up with a load-balancing switch in front of murder front-end machines which in turn are in front of the backend mailstore machines. mupdate has to be run and mail delivery has to be directed to the correct machine, etc. Moreover, with the murder design, backend machines would have to be clustered anyway (2-node clusters probably) to eliminate the back-end machines as SPOFs. With a cluster configuration, all machines would run sendmail and master/imapd and offer exactly the same service. A load-balancing switch is in front of the cluster and any cluster machine can go down without losing any service. I can't see any functionality that would be lost but I'm pretty sure that the cluster solution will be more expensive even though the expense of buying murder front-end machines would be eliminated. So, my first question is: has anyone implemented a Veritas-based Cyrus service? And my second question is: is there something I'm missing with regard to the Veritas cluster being a much simpler configuration to troubleshoot and operate and a much stronger configuration in terms of availability? TIA for all input. Ben Carter University of Pittsburgh/CSSD [EMAIL PROTECTED] 412-624-6470
Re: Highly available and scalable Cyrus configurations
--On Tuesday, October 07, 2003 6:43 PM -0400 Rob Siemborski [EMAIL PROTECTED] wrote:r On Tue, 7 Oct 2003, Ben Carter wrote: and an MTA on each cluster member, because with the murder configuration, you probably have to end up with a load-balancing switch in front of murder front-end machines which in turn are in front of the This is not the case. Simple DNS round-robining is sufficient. How is simple DNS round-robin sufficient? More than one MUA caches the DNS lookups for its mail servers. I know that at least one version of Netscape Communicator used to. It probably still does. Also, some webmail clients do. So if a front-end is down or has not network connectivity, one gets hung MUAs. design, backend machines would have to be clustered anyway (2-node clusters probably) to eliminate the back-end machines as SPOFs. Well, the aggregator isn't solving the HA problem. It is doing its best to mitigate it however (via partial failure modes). I know. The aggregator is better for us for at least one other reason: we can use our existing SCSI storage arrays. Veritas does not support clustering with SCSI (changing the host SCSI initiator IDs to be different). And my second question is: is there something I'm missing with regard to the Veritas cluster being a much simpler configuration to troubleshoot and operate and a much stronger configuration in terms of availability? Well its simpler unless it breaks or has poor performance. OK. I thought so. I'm not familiar with the Veritas cluster (we have had good experience with vxfs on our backends however). You also probably want to be sure you're not replaceing one single point of failure (a backend) with another (the cluster). The key about the usability of the filesystem is that file locks need to be obeyed throughout the entire cluster, and mmap() needs to be efficient (and able to deal with read() and write() being called on that file at the same time). That the locking needs to work is clear. I did not think of the mmap() vs read/write issues. Thanks. Also, the murder does get you one performance advantage that the veritas cluster (even correctly implemented) does not: it provides a large number of read-only copies of the mailbox list that clients can use for LIST operations, and not interfere with updates of the master copy. With the size of mailbox lists on systems that are considering HA options, or moving to an aggregator configuration, this is, in fact, a serious concern to think about. That's another good point. However, is this still an issue if one does not select the flat file format for the mailboxes file? Has anyone implemented Cyrus IMAP with any type of clustering softwarethat you know of? Thanks, Ben -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper Ben Carter Computing Services System Development University of Pittsburgh [EMAIL PROTECTED] 412-624-6470
