>I'm no ubik engineer, but as far as I understand it, the protocol was not >designed for even numbers of participating servers. For best results, three >or five servers seem to be optimum.
There is a lot of misinformation about Ubik out there; the voting protocol is actually not complicated, it's just not documented well. Looking at the source code is even more confusing. So, let me clear up some misconceptions: - It's not really that an odd number is optimum; it's just that you're wasting a server with an even number. Why? Well, the Ubik voting requires a majority number of servers to win an election; if there are 4 servers and only two are available, then that's not a majority. So with 4 servers, you can lose only one server and still maintain quorum (same as with three). You need 5 servers to be able to lose two of them. Now, there is an extra wrinkle here ... the "best" server (lowest numbered) gets an extra vote. So in a 4 server configuration, you can actually lose two and maintain quorum ... as long as one of the two isn't the "best" server. But with five servers, you can lose ANY two. But the protocol works fine with two, or three, or four, or five. There is NO magic here. >What I definitely whitnessed is that servers in a cell configured with two >servers take more than a minute to elect a sync site after server restarts. >Three servers are supposed to make it in an instant. This is one of those mostly-not-true statements that has a bit of truth in it. The exact details: - When brought up, a database server will not vote YES for anyone for 75 seconds. This is inviolate. It doesn't matter if there are two, three, or 100 database servers. If you bring up all your servers cold, at the same time, it will take at least 75 seconds for a quorum election. - If you have two database servers and you only restart the "best" server (note: in a two database server cell, only the "best" server can ever be elected as master), a new election will take 75 seconds. Why? Because you have to wait for the best server to be able to vote for itself; without that vote, there is not a majority. - If you have three (or more) database servers and you only restart the current master, a successful election will happen almost instantly. Why? Because all of the servers that are still up will still vote YES for the master; the master's own YES vote is not necessary. But note this only applies if all of the other servers are still running. If, for example, you rebooted the master and if it took longer than 75 seconds for the master to restart, then what will likely happen is a new master will be elected. Getting back to the original poster's question ... by far the most common problem I have seen with Ubik is bad time synchronization. All of your database servers must be synched up time-wise (the protocol depends on timestamps). It doesn't need to be femtosecond accuracy; the protocol defines MAXSKEW as 10 seconds. If your database servers are accessable via the Internet, we could take a look at them via udebug. Really, there are only a few things that can go wrong; of all of the pieces of AFS, I think Ubik is one of the most bulletproof. --Ken _______________________________________________ OpenAFS-info mailing list [email protected] https://lists.openafs.org/mailman/listinfo/openafs-info
