RE: A beginner question about Murder
So we use about ten common Intel based servers for BEs (sized to support loss of 2/3 servers) and store data on NAS. Users are active and filer is about tens of thousand nfsop/s at the busy hour. About 10% to 20% of the users connect at least one time a day, globally 4 million connections per day. These are quite massive figures -- I'm thoroughly impressed. I am assuming that your back-ends experience 50,000-100,000 concurrent IMAP connections during certain parts of the day. This means that each BE takes 5,000-10,000 parallel connections. Can an ordinary Intel server handle such loads? What's the CPU and RAM of each? We don't have to handle such load. Most of end-users connect on pop3d (we provide a pop3 optimizer to reduce I/O, see http://www.mail-archive.com/cyrus-de...@lists.andrew.cmu.edu/msg01194.html) , others connect on the webmail (our webmail uses disconnected imapd sessions and is optimized thanks to Cyrus statuscache db). Also, how many NAS servers/appliances are being used? I would have thought that something like NFS would be too slow for such heavy IO loads, but evidently I'm wrong. One huge is enough and cheaper than a SAN. Also we have very experienced system admins to tune the filer ! Ce message et les pièces jointes sont confidentiels et réservés à l'usage exclusif de ses destinataires. Il peut également être protégé par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir immédiatement l'expéditeur et de le détruire. L'intégrité du message ne pouvant être assurée sur Internet, la responsabilité du groupe Atos Origin ne pourra être recherchée quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'expéditeur ne donne aucune garantie à cet égard et sa responsabilité ne saurait être recherchée pour tout dommage résultant d'un virus transmis. This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos Origin group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
RE: A beginner question about Murder
Hi, we use a modified traditional murder, i.e. without murder daemon, to host more than 2 million mailboxes (dozen million entries in mboxlist with folders) We started to host 200.000 mailboxes on the traditional murder, configured with berkeley as cyrusdb backend due to lot of corruptions with default skiplist format (cyrus 2.3.9). However we had a lot of locks and contentions with berkeleydb, so we migrated later to cyrus 2.3.13 on a skiplist backend. Unfortunately the more we hosted mailboxes the more we had production issues. Indeed database resync is not incremental and servers restart were increasingly longer. Moreover we had instability with skiplist backend around 7.8 million entries and failed to reach 8 million entries. Finally we chose to develop a customized mysql backend. Front-ends and back-ends are all connected to the same DB but SQL results are rewritten with plugins to make them happy (return mailbox type 1 for proxy en mailbox type 0 for backend). Now there is no more latency or issue and mysql allows us to host more several dozens million of entries. So, we are totally murderless. FEs are scalable thanks to mysql replication and BEs are almost high available (our mysql failover is read-only and don't permit acl or folder management, but it is acceptable for us). We developed our owns mysql backend and we can share the source code if someone is interested (perhaps need some clean-up and code rewrite to adapt to the official sql backend) Sébastien -Message d'origine- De : info-cyrus-bounces+sebastien.michel=atosorigin@lists.andrew.cmu.edu [mailto:info-cyrus-bounces+sebastien.michel=atosorigin@lists.andrew.cmu.edu] De la part de Wesley Craig Envoyé : jeudi 9 septembre 2010 20:16 À : Andrew Morgan Cc : Shuvam Misra; info-cyrus@lists.andrew.cmu.edu Objet : Re: A beginner question about Murder On 08 Sep 2010, at 16:41, Andrew Morgan wrote: Unfortunately, I've never setup a unified Murder, so I don't fully understand what the advantages and disadvantages of it compared to a traditional Murder. Maybe someone else can jump in here with their experiences. The main advantage is that BEs can proxy for other BEs, no FEs are required. In m observation, the FEs are typically CPU bound, while the BEs are typically IO bound. Since the BEs have an excess of CPU, it makes some sense to combine the two roles. Other than the bugs one finds in new code, unified is logically superior to traditional and in the long run traditional should be retired. In the future, I'd like to add unified replication, masterless murder, and auto failover. In my ideal minimum system, there are two machines, replicating to each other, providing high availability, proxying, etc. :wes Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ Ce message et les pièces jointes sont confidentiels et réservés à l'usage exclusif de ses destinataires. Il peut également être protégé par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir immédiatement l'expéditeur et de le détruire. L'intégrité du message ne pouvant être assurée sur Internet, la responsabilité du groupe Atos Origin ne pourra être recherchée quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'expéditeur ne donne aucune garantie à cet égard et sa responsabilité ne saurait être recherchée pour tout dommage résultant d'un virus transmis. This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos Origin group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
RE: A beginner question about Murder
Dear Michel, we use a modified traditional murder, i.e. without murder daemon, to host more than 2 million mailboxes (dozen million entries in mboxlist with folders) Wow, that's some figure. Care to share some details with the list? What kind of hardware (both for servers and storage), what performance observations, etc? How many of these 2 million mailboxes see at least one login a day? Shuvam Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
RE: A beginner question about Murder
Hi, yes I can share some little details. So we use about ten common Intel based servers for BEs (sized to support loss of 2/3 servers) and store data on NAS. Users are active and filer is about tens of thousand nfsop/s at the busy hour. About 10% to 20% of the users connect at least one time a day, globally 4 million connections per day. Sébastien -Message d'origine- De : info-cyrus-bounces+sebastien.michel=atosorigin@lists.andrew.cmu.edu [mailto:info-cyrus-bounces+sebastien.michel=atosorigin@lists.andrew.cmu.edu] De la part de Shuvam Misra Envoyé : jeudi 16 septembre 2010 10:48 À : info-cyrus@lists.andrew.cmu.edu Objet : RE: A beginner question about Murder Dear Michel, we use a modified traditional murder, i.e. without murder daemon, to host more than 2 million mailboxes (dozen million entries in mboxlist with folders) Wow, that's some figure. Care to share some details with the list? What kind of hardware (both for servers and storage), what performance observations, etc? How many of these 2 million mailboxes see at least one login a day? Shuvam Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ Ce message et les pièces jointes sont confidentiels et réservés à l'usage exclusif de ses destinataires. Il peut également être protégé par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir immédiatement l'expéditeur et de le détruire. L'intégrité du message ne pouvant être assurée sur Internet, la responsabilité du groupe Atos Origin ne pourra être recherchée quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'expéditeur ne donne aucune garantie à cet égard et sa responsabilité ne saurait être recherchée pour tout dommage résultant d'un virus transmis. This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos Origin group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
RE: A beginner question about Murder
So we use about ten common Intel based servers for BEs (sized to support loss of 2/3 servers) and store data on NAS. Users are active and filer is about tens of thousand nfsop/s at the busy hour. About 10% to 20% of the users connect at least one time a day, globally 4 million connections per day. These are quite massive figures -- I'm thoroughly impressed. I am assuming that your back-ends experience 50,000-100,000 concurrent IMAP connections during certain parts of the day. This means that each BE takes 5,000-10,000 parallel connections. Can an ordinary Intel server handle such loads? What's the CPU and RAM of each? Also, how many NAS servers/appliances are being used? I would have thought that something like NFS would be too slow for such heavy IO loads, but evidently I'm wrong. regards, Shuvam Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
Quoting Shuvam Misra shuvam.mi...@merceworld.com: This means that there's no (easy) way to compile the murder master to keep its config and meta-data files in a different directory from the one used by the back-end cyrus IMAP daemon, I guess. Am I right? No. You only have to add -C option in cyrus.conf to point to an extra impad.conf for backend, frontend and mupdate and specify the ip in the listen option if a service is running on frontend and backend (beware that the service name must be unique. In each imapd.conf you can set configdirectory to point to the correct directory. Shuvam Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ M.MengeTel.: (49) 7071/29-70316 Universität Tübingen Fax.: (49) 7071/29-5912 Zentrum für Datenverarbeitung mail: michael.me...@zdv.uni-tuebingen.de Wächterstraße 76 72074 Tübingen smime.p7s Description: S/MIME Signatur Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
Hi, The tricky part is - there should be a master, and it should probably either be chosen by some sort of election between the active servers (here lies high-availability heartbeat magic) or by IP address allocation. Anybody know if avahi and friends can help here? spread (http://www.spread.org/), as exemplified by Wackamole (http://www.backhand.org/wackamole/) is good for distributed heartbeat and election algorithms. It takes more work to make it reliable for heavy data transfer, but Cyrus has solved that part of the problem already. Regards, @ndy -- andy...@ashurst.eu.org http://www.ashurst.eu.org/ 0x7EBA75FF Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
On Wed, Sep 08, 2010 at 01:41:26PM -0700, Andrew Morgan wrote: Unfortunately, I've never setup a unified Murder, so I don't fully understand what the advantages and disadvantages of it compared to a traditional Murder. Maybe someone else can jump in here with their experiences. And while they're at it, they could hopefully give an opinion on whether there's any point keeping traditional Murder from a technical standpoint, i.e. if it would hurt to make all Murder unified and if people choose to lay out their servers in the traditional manner it would still just work. I for one, a newbie to Cyrus, would vote for a design which allows all-in-one servers, and allows separation if needed. Basically, localhost should always be a valid network destination for all types of network communication based systems. Why is this a problem with current Cyrus Murder setups? What internal details prevent a front-end server, mupdate server, and back-end server from coexisting on the same physical system? Is it that there's no facility in the back-end server feature to make it listen only on localhost:imap and not on *:imap? If that's the only problem, I am sure it'll be a simple patch to fix. But I guess the problem is bigger than this. Shuvam Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
Dear Andrew, In a traditional Cyrus Murder (not a unified Murder), there are 3 roles: 1. backends - these store email 2. frontends - these proxy incoming connections to the correct backend 3. mupdate master - maintains the list of mailboxes in the Murder There can only be 1 mupdate master process. I'm not positive if you can run it on a backend or frontend server, or if it must be running on a separate server. Thanks a lot -- this block alone was very helpful for a newbie like me. Sets my terminology right and gives me a clear picture of the building blocks. When a new mailbox is created on a backend, the backend first checks the mupdate master to make sure the mailbox name is not already in use. Then it reserves the name, creates the mailbox, and registers the mailbox with the mupdate master. Does this mean that all cyradm-type admin connections must connect to one or other of the back-end servers? Can I get admin tasks done by connecting to one of the front-end servers too? Shuvam Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
Quoting Shuvam Misra shuvam.mi...@merceworld.com: On Wed, Sep 08, 2010 at 01:41:26PM -0700, Andrew Morgan wrote: Unfortunately, I've never setup a unified Murder, so I don't fully understand what the advantages and disadvantages of it compared to a traditional Murder. Maybe someone else can jump in here with their experiences. And while they're at it, they could hopefully give an opinion on whether there's any point keeping traditional Murder from a technical standpoint, i.e. if it would hurt to make all Murder unified and if people choose to lay out their servers in the traditional manner it would still just work. I for one, a newbie to Cyrus, would vote for a design which allows all-in-one servers, and allows separation if needed. Basically, localhost should always be a valid network destination for all types of network communication based systems. Why is this a problem with current Cyrus Murder setups? What internal details prevent a front-end server, mupdate server, and back-end server from coexisting on the same physical system? Is it that there's no facility in the back-end server feature to make it listen only on localhost:imap and not on *:imap? If that's the only problem, I am sure it'll be a simple patch to fix. But I guess the problem is bigger than this. In the traditional Murder it is possible to run two or more Cyrus instances on one host, but you must use additional IP-Addresses to separate frontend, backend and mupdate server. In the unified setup you don't need the additional IPs, but there are some known bugs in the unified murder. http://bugzilla.cyrusimap.org/bugzilla3/show_bug.cgi?id=2814 http://bugzilla.cyrusimap.org/bugzilla3/show_bug.cgi?id=2915 http://bugzilla.cyrusimap.org/bugzilla3/show_bug.cgi?id=2973 http://bugzilla.cyrusimap.org/bugzilla3/show_bug.cgi?id=2976 http://bugzilla.cyrusimap.org/bugzilla3/show_bug.cgi?id=3110 Shuvam Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ M.MengeTel.: (49) 7071/29-70316 Universität Tübingen Fax.: (49) 7071/29-5912 Zentrum für Datenverarbeitung mail: michael.me...@zdv.uni-tuebingen.de Wächterstraße 76 72074 Tübingen smime.p7s Description: S/MIME Signatur Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
Le 09/09/2010 01:48, Jeroen van Meeuwen (Kolab Systems) a écrit : Clement Hermann (nodens) wrote: In traditional murder (no autocreate/autosieve patch), the murder process can run on a frontend. However, it cannot run on a backend. We have a webmail running on our murder (2.2.x) server, and it uses localhost as imap server, so it acts as a frontend. However, we don't use autocreate or autosieve, so I couldn't says if it is the same on a patched setup. When you say traditional murder, do you also mean one single mupdate server? Yes. I'm sure it's possible, the documentation states it as well. I just haven't managed to actually get it implemented (with the patches that have been shipped for about 5-6 years, in at least 4 generations of Enterprise Linux products and 13 Fedora releases). That's not implemented besides maintaining the actual functionality of the autocreation of mailboxes or sieve scripts or implementation details thereof; the cyrus master process just wouldn't start. No problem here in using proxyd and mupdate -m (again, debian 2.2 stock packages without additionnal patches). The options mupdate_server, mupdate_authname, and mupdate_password are simply commented in imapd.conf. Cheers, -- Clement Hermann (nodens) Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
On Sep 9, 2010, at 2:22, Shuvam Misra shuvam.mi...@merceworld.com wrote: Does this mean that all cyradm-type admin connections must connect to one or other of the back-end servers? Can I get admin tasks done by connecting to one of the front-end servers too? Most admin tasks are proxied by the frontends. If defaultserver or serverlist aren't set, create must specify the backend if connected to a frontend. :wes Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
On Sep 8, 2010, at 17:17, Jeroen van Meeuwen (Kolab Systems) vanmeeu...@kolabsys.com wrote: - In a tradition Murder setup the master update server cannot be combined with a backend or frontend server. The murder master can also be a frontend server, but not a backend server. The mailboxes.db in a traditional murder uses different formats on frontends and backend. Any frontend may be trivially promoted to the murder master roll. :wes Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
On Sep 9, 2010, at 2:20, Shuvam Misra shuvam.mi...@merceworld.com wrote: What internal details prevent a front-end server, mupdate server, and back-end server from coexisting on the same physical system? Is it that there's no facility in the back-end server feature to make it listen only on localhost:imap and not on *:imap? No. Mostly it's a format difference between backend and frontends/ murder master. The unified configuration solves many of those problems. All in one is not a murder btw, and Cyrus does a single stand alone IMAP server just fine. :wes Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
Does this mean that all cyradm-type admin connections must connect to one or other of the back-end servers? Can I get admin tasks done by connecting to one of the front-end servers too? Most admin tasks are proxied by the frontends. If defaultserver or serverlist aren't set, create must specify the backend if connected to a frontend. Okay, got it. thanks, Shuvam Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
On 08 Sep 2010, at 16:41, Andrew Morgan wrote: Unfortunately, I've never setup a unified Murder, so I don't fully understand what the advantages and disadvantages of it compared to a traditional Murder. Maybe someone else can jump in here with their experiences. The main advantage is that BEs can proxy for other BEs, no FEs are required. In m observation, the FEs are typically CPU bound, while the BEs are typically IO bound. Since the BEs have an excess of CPU, it makes some sense to combine the two roles. Other than the bugs one finds in new code, unified is logically superior to traditional and in the long run traditional should be retired. In the future, I'd like to add unified replication, masterless murder, and auto failover. In my ideal minimum system, there are two machines, replicating to each other, providing high availability, proxying, etc. :wes Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
On Thu, Sep 09, 2010 at 02:15:52PM -0400, Wesley Craig wrote: On 08 Sep 2010, at 16:41, Andrew Morgan wrote: Unfortunately, I've never setup a unified Murder, so I don't fully understand what the advantages and disadvantages of it compared to a traditional Murder. Maybe someone else can jump in here with their experiences. The main advantage is that BEs can proxy for other BEs, no FEs are required. In m observation, the FEs are typically CPU bound, while the BEs are typically IO bound. Since the BEs have an excess of CPU, it makes some sense to combine the two roles. Other than the bugs one finds in new code, unified is logically superior to traditional and in the long run traditional should be retired. Sounds good to me. Let's have a look at those bugs and see how bad they are! If we can make one murder to rule them all I'd be happy :) In the future, I'd like to add unified replication, masterless murder, and auto failover. In my ideal minimum system, there are two machines, replicating to each other, providing high availability, proxying, etc. Yes, this. Absolutely. The replication code is pretty safe for multi-master in my branch already. At least for mailboxes. Sieve, Seen and Subs are somewhat trickier. I think the only really safe way is to keep deletion entries around and replicate those too so you can tell the difference between a creation at one and and a deletion at the other! Anyway - I've been thinking about this a bit. Some of it's not too hard, some is trickier. In particular I think each mailbox (and by extention each user - maybe based on their INBOX...) needs to have a list of machines it should be replicated on and a current primary which are sent to the murder. The tricky part is - there should be a master, and it should probably either be chosen by some sort of election between the active servers (here lies high-availability heartbeat magic) or by IP address allocation. Anybody know if avahi and friends can help here? I really like this idea - particularly if servers can have almost-auto- discovery. It would be great if you could just run up a bunch of Cyrus servers out of the box and have them talk to each other. Bron ( and I really like the idea of being able to replicate a subset of users - because it would let you set up a home replica that talks to your ISP/Work/Uni server and gives you super-fast access while still keeping full IMAP semantics! ) Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
On 09/08/2010 11:41 AM, Shuvam Misra wrote: I've never set up Murder (I'm sure it's obvious) It seems obvious to me, destroy all the evidence. (snicker) -- Sincerely, John Thomas Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
What internal details prevent a front-end server, mupdate server, and back-end server from coexisting on the same physical system? Is it that there's no facility in the back-end server feature to make it listen only on localhost:imap and not on *:imap? No. Mostly it's a format difference between backend and frontends/ murder master. The unified configuration solves many of those problems. All in one is not a murder btw, and Cyrus does a single stand alone IMAP server just fine. This means that there's no (easy) way to compile the murder master to keep its config and meta-data files in a different directory from the one used by the back-end cyrus IMAP daemon, I guess. Am I right? Shuvam Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
Yes, this. Absolutely. The replication code is pretty safe for multi-master in my branch already. At least for mailboxes. Sieve, Seen and Subs are somewhat trickier. I think the only really safe way is to keep deletion entries around and replicate those too so you can tell the difference between a creation at one and and a deletion at the other! Aren't we really talking single-master at the mailbox level? The problems of true multi-master are non-trivial, IMHO, but if we are running a pair of Cyrus stacks replicating on each other, each mailbox will have its master copy only on one server, and will replicate on the other. The trick then will be to separate out the IMAP daemon and the sync_server to make them work on separate partitioned areas of the same message store and meta-data store. The IMAP daemon will only serve those parts which have local store as master, and sync-server will only update those areas for which it is acting as replica server. If the buddy server fails, then I should be able to get the remaining server into full-serving mode by switching config files, shutting down sync-server, and allowing the IMAP daemon to serve data from the entire (now undivided) store. Please correct me if my understanding is wrong? Will this be the right way to proceed? Shuvam Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
On Fri, Sep 10, 2010 at 09:37:29AM +0530, Shuvam Misra wrote: Yes, this. Absolutely. The replication code is pretty safe for multi-master in my branch already. At least for mailboxes. Sieve, Seen and Subs are somewhat trickier. I think the only really safe way is to keep deletion entries around and replicate those too so you can tell the difference between a creation at one and and a deletion at the other! Aren't we really talking single-master at the mailbox level? The problems of true multi-master are non-trivial, IMHO, but if we are running a pair of Cyrus stacks replicating on each other, each mailbox will have its master copy only on one server, and will replicate on the other. ... ish. I should write this up in more detail - but it's _fairly_ safe even against changes on both ends. I wrote up the UID promotion logic. The trick then will be to separate out the IMAP daemon and the sync_server to make them work on separate partitioned areas of the same message store and meta-data store. The IMAP daemon will only serve those parts which have local store as master, and sync-server will only update those areas for which it is acting as replica server. Actually I plan to make sync_server update anything. It won't be told about stuff that isn't getting changes from the other end. If the buddy server fails, then I should be able to get the remaining server into full-serving mode by switching config files, shutting down sync-server, and allowing the IMAP daemon to serve data from the entire (now undivided) store. Ideally it should just be a matter of creating or removing a single file on the filesystem without needing to restart anything. A server X is down file, which would pause replication attempts and cause the next replica to be chosen for all mailboxes mastered on that host. Please correct me if my understanding is wrong? Will this be the right way to proceed? Hey - it doesn't even exist yet! But my idea is to have multiple target replication in a fairly flexible way, and I'm leaning towards having both for all mailboxes replication target in the config file and per mailbox replication target in the mailboxes.db. (possibly the one in the imapd.conf should actually just add itself to any mailbox where it isn't already present - so that the change can then propagate to every other server in the murder, and they can all know where the mailbox might be found) hmm... Bron. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
Hello, The cyrus murder/aggregator finds its machines by their names, not ip. You have to have dns records or all /etc/hosts configured. You may use virtual machines for the mupdate master, for example, at your servers. Regards. Andre Felipe Machado Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
On Thu, 9 Sep 2010, Shuvam Misra wrote: Dear all, If I have, say, three IMAP servers each hosting a few thousand mailboxes, and I want to aggregate all of them for the IMAP client, I'll run Murder on one of the servers. 1. Can I run Murder on one of the back-end servers? If yes, it will act as an aggregator for incoming connections from clients, and it will also serve a part of the mailboxes. This means that the murder daemon (whatever it is called) will sometimes talk to the local IMAP daemon. Does this work? 2. If yes to the last question, then can I run Murder on all my three IMAP daemons? This way, I'll be able to distribute the load of both aggregating and mailbox serving across all my servers, if I can tell my IMAP clients to distribute their connections across these three Murder servers. Can this sort of setup work? If yes, is this is a good idea, compared to dedicated aggregator hardware and dedicated mailbox-serving servers? I've never set up Murder (I'm sure it's obvious) and I'm wondering about what architecture works well. We're trying to build some multi-IMAP-server setups for one or two of our customers. In a traditional Cyrus Murder (not a unified Murder), there are 3 roles: 1. backends - these store email 2. frontends - these proxy incoming connections to the correct backend 3. mupdate master - maintains the list of mailboxes in the Murder There can only be 1 mupdate master process. I'm not positive if you can run it on a backend or frontend server, or if it must be running on a separate server. We run a traditional Cyrus Murder here. We have 3 backend servers (the number is determined by performance and capacity requirements). We have 3 frontend servers (for redundancy and performance) that sit behind a ServerIron hardware load balancer and all answer for imap.onid.oregonstate.edu, which is the IMAP server name all of our clients use. We have 1 standalone mupdate master server. The backend servers register all of their mailboxes with the mupdate master, so that it has a complete list of all the mailboxes and their locations in the Murder. The frontend servers read and cache the list of mailboxes from the mupdate master, so that they can proxy incoming IMAP client connections to the correct backend. When a new mailbox is created on a backend, the backend first checks the mupdate master to make sure the mailbox name is not already in use. Then it reserves the name, creates the mailbox, and registers the mailbox with the mupdate master. Unfortunately, I've never setup a unified Murder, so I don't fully understand what the advantages and disadvantages of it compared to a traditional Murder. Maybe someone else can jump in here with their experiences. Andy Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
On Wed, Sep 08, 2010 at 01:41:26PM -0700, Andrew Morgan wrote: Unfortunately, I've never setup a unified Murder, so I don't fully understand what the advantages and disadvantages of it compared to a traditional Murder. Maybe someone else can jump in here with their experiences. And while they're at it, they could hopefully give an opinion on whether there's any point keeping traditional Murder from a technical standpoint, i.e. if it would hurt to make all Murder unified and if people choose to lay out their servers in the traditional manner it would still just work. Bron ( going to play with this very, very soon - I haven't used murder at all - but I need to make sure it still works in my code branch! ) Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
On Wed, Sep 08, 2010 at 11:17:00PM +0200, Jeroen van Meeuwen (Kolab Systems) wrote: - For autocreate/autosieve (patches for which Cyrus is not upstream but they are shipped with Fedora and Red Hat Enterprise Linux packages), the frontend servers must be disabled for local direct delivery through the lmtp proxy, and instead relay through the backend server's MTA for autocreate to create the mailbox on a backend server (and not a frontend server which would then loop back to itself). The same goes for autocreate on login, which would cause the frontend to create a mailbox on the local default partition rather then on one of the backends in the Murder. Sounds to me like a case for fixing the autocreate/autosieve patches! Bron ( hey - what timezone are you in, and do you use instant messaging? ) Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
Bron Gondwana wrote: On Wed, Sep 08, 2010 at 11:17:00PM +0200, Jeroen van Meeuwen (Kolab Systems) wrote: - For autocreate/autosieve (patches for which Cyrus is not upstream but they are shipped with Fedora and Red Hat Enterprise Linux packages), the frontend servers must be disabled for local direct delivery through the lmtp proxy, and instead relay through the backend server's MTA for autocreate to create the mailbox on a backend server (and not a frontend server which would then loop back to itself). The same goes for autocreate on login, which would cause the frontend to create a mailbox on the local default partition rather then on one of the backends in the Murder. Sounds to me like a case for fixing the autocreate/autosieve patches! Bron ( hey - what timezone are you in, and do you use instant messaging? ) Hey, I'm in UTC+2 sometimes, and UTC+1 some other times, and then DST and such and so forth... the Netherlands, Switzerland and the UK ;-) My Google Talk username is kana...@gmail.com Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
Bron Gondwana wrote: On Wed, Sep 08, 2010 at 11:17:00PM +0200, Jeroen van Meeuwen (Kolab Systems) wrote: - For autocreate/autosieve (patches for which Cyrus is not upstream but they are shipped with Fedora and Red Hat Enterprise Linux packages), the frontend servers must be disabled for local direct delivery through the lmtp proxy, and instead relay through the backend server's MTA for autocreate to create the mailbox on a backend server (and not a frontend server which would then loop back to itself). The same goes for autocreate on login, which would cause the frontend to create a mailbox on the local default partition rather then on one of the backends in the Murder. Sounds to me like a case for fixing the autocreate/autosieve patches! Uh oh, I forgot that went back to the list as well. Ohw well. To answer the remark; yes it does mean that. I'm going to have to do that / make that happen, and other miscellaneous stuff. It is well-known the autocreate patches are not really compatible with a traditional Murder setup. I have a branch in my GIT repository, but I'm not fully up to speed as to the exact implementation details yet. I'm not at all that well a coder anyways. Maybe we can discuss in a chat session sometime? Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
Le 08/09/2010 23:17, Jeroen van Meeuwen (Kolab Systems) a écrit : Andrew Morgan wrote: In a traditional Cyrus Murder (not a unified Murder), there are 3 roles: 1. backends - these store email 2. frontends - these proxy incoming connections to the correct backend 3. mupdate master - maintains the list of mailboxes in the Murder There can only be 1 mupdate master process. I'm not positive if you can run it on a backend or frontend server, or if it must be running on a separate server. In my test setup (internal Wiki document attached licensed CC-BY-SA), which to date is still a work in progress, it appeared to me; - In a tradition Murder setup the master update server cannot be combined with a backend or frontend server. - For autocreate/autosieve (patches for which Cyrus is not upstream but they are shipped with Fedora and Red Hat Enterprise Linux packages), the frontend servers must be disabled for local direct delivery through the lmtp proxy, and instead relay through the backend server's MTA for autocreate to create the mailbox on a backend server (and not a frontend server which would then loop back to itself). The same goes for autocreate on login, which would cause the frontend to create a mailbox on the local default partition rather then on one of the backends in the Murder. In traditional murder (no autocreate/autosieve patch), the murder process can run on a frontend. However, it cannot run on a backend. We have a webmail running on our murder (2.2.x) server, and it uses localhost as imap server, so it acts as a frontend. However, we don't use autocreate or autosieve, so I couldn't says if it is the same on a patched setup. Cheers, -- Clement Hermann (nodens) - L'air pur ? c'est pas en RL, ça ? c'est pas hors charte ? Jean in L'Histoire des Pingouins, http://tnemeth.free.fr/fmbl/linuxsf/ Vous trouverez ma clef publique sur le serveur public pgp.mit.edu. Please find my public key on the public keyserver pgp.mit.edu. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: A beginner question about Murder
Clement Hermann (nodens) wrote: In traditional murder (no autocreate/autosieve patch), the murder process can run on a frontend. However, it cannot run on a backend. We have a webmail running on our murder (2.2.x) server, and it uses localhost as imap server, so it acts as a frontend. However, we don't use autocreate or autosieve, so I couldn't says if it is the same on a patched setup. When you say traditional murder, do you also mean one single mupdate server? I'm sure it's possible, the documentation states it as well. I just haven't managed to actually get it implemented (with the patches that have been shipped for about 5-6 years, in at least 4 generations of Enterprise Linux products and 13 Fedora releases). That's not implemented besides maintaining the actual functionality of the autocreation of mailboxes or sieve scripts or implementation details thereof; the cyrus master process just wouldn't start. Thank you in advance, Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
