On Fri, 17 Sep 2004, Paul Dekkers wrote:
Date: Fri, 17 Sep 2004 08:25:26 +0200 From: Paul Dekkers <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: Funding Cyrus High Availability
Hi,
Eric S. Pulley wrote:
Question: Are people looking at this as both redundancy and performance, or just redundance?
Cyrus performs pretty well already. Background redundancy would be awesome. Especially if we had control over when the syncing process occurred either via time interval or date/time.
I would say not at an interval but as soon as there is an action performed on one mailbox, the other one would be pushed to do something. I believe that is called rolling replication.
I would not be really happy with a interval synchronisation. It would make it harder to use both platforms at the same time, and that is what I want as well. So there is a little-bit of load-balancing involved, but more and more _availability_.
Being able to use both platforms at the same time maybe implies that there is either no master/slave role or that this is auto-elected between the two and that this role is floating...
right, but there are already tools freely available on most platforms to do the election and changing of the role (by switching between config files and restarting the master) what is currently lacking is any ability to do the master/slave role. once we have that it's just a little scripting to tie just about any failover software in to make it automatic.
one thing we need to watch out for here is that we don't set an impossible/unreasonable goal. don't try to solve every problem and add every availablity feater you can imagine all at once. instead let's look at the building blocks that are needed and identify what's currently not available.
currently we have murder which will spread the load across multiple machines.
currently we have many tools available to detect a server failure and run local scripts to reconfigure machines (HACMP on AIX, hearbeat for Linux, *BSD, Solaris, etc)
what we currently do not have is any ability to have one mailstore updated to match changes in another one.
once we have that ability there are many things that can be built by glueing togeather existing code. once we have a bit of experiance with people actually useing these features it will then be obvious which features need better integration with Cyrus and which make sense to remain seperate.
I also would not be really satisfied with interval synchronisation as the only choice.
I think we need something where the primary mailstore pushes a record of it's changes to the secondary mailstore
This can then be tweaked in several directions.
1. locking can be added so that the primary doesn't complete it's command until the secondary says it has a permanent record of the change (two-phase commit or a reasonable facimily of such)
2. batch up the changes until they hit some threshold (size or time or combination) and then send a batch of changes all at once
3. recongnise it's own changes to gain the ability to push updates in both directions at the same time (true two-phase commit with bi-directional replication, some horribile performance pathalogical cases, but attractive in some cases)
or other varients
but these all share the same common need
the ability for the master to output all it's changes and the ability for a slave to read such changes and update itself to match
the nice thing is that with IMAP much of the data needed is already output, you could do a first approximation of this with a client that opened a seperate connection to every folder on the primary server and just sat watching for server messages and whenever it saw an update send the matching command to the slave (fetching the full data as needed to get all the info). this obviously won't scale to any reasonalbe size, but this means that most of what's needed is already identified so the core could be just a common output of the exisitng messages with a little more data (mailbox and folder in most cases, message contents in a few)
let's get these small, but critical pieces done and then we can grow and experiment from there.
David Lang
Paul
--- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
-- There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies. -- C.A.R. Hoare --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
