Lars Marowsky-Bree wrote:
> On 2007-07-13T13:34:59, Alan Robertson <[EMAIL PROTECTED]> wrote:
>
>> Here's the bugzilla for that capability:
>> http://old.linux-foundation.org/developer_bugzilla/show_bug.cgi?id=1497
>>
>> This would greatly improve the situation for 2-node clusters and in most
>> cases would eliminate the possibility of mutual-STONITH.
>
> This is what people are currently using quorumd for, and I actually
> think it is the smarter choice. (As in - the quorumd process has smarts
> to decide, whereas a ping node is just dumb and might return pings to
> both, in particular in the case of broken switches and stuff.)
_Of course_ quorumd does that job better - that's why I designed it -
over Lars' violent objections ;-). It's also a lot more trouble to set
up. The proposed ping quorum tiebreaker is much better than nothing -
which is exactly what most people are using, and it's trivial to set up.
It provides a perfect fill-in-the-gap solution for people with a small
number of clusters.
Since it's STONITH that protects the data, all this protects you from is
mutual STONITH in some rare cases. No data would be jeopardized by
using such a plugin.
Quorum modules can be used in sequence, and each returns one of three
values:
HAVEQUORUM
QUORUMTIE
NOQUORUM
Only if a module returns QUORUMTIE is the next module consulted. So,
the subsequent ones are used to "break the tie" given by previous
modules. If the last module returns QUORUMTIE, it is treated as NOQUORUM.
So, for the rest of the crowd who doesn't know what's being discussed
here and the history behind it, here's a little more info about the
different quorum modules (all but one are implemented already). They're
mostly listed in order of decreasing "quality" and also decreasing
complexity (except for majority vote, which is really simple)
MajorityVote - returns TRUE if nactive >= ((nnode/2)+1). This
is a very traditional majority quorum method. It can
only produce false positives when the arithmetic fails,
or if the underlying membership method fails to give
a valid result. It returns QUORUMTIE if
nactive*2 == nnode.
Quorumd - a software version of SCSI reserve (or equivalent)
used as a sort of quorum tiebreaker. Very useful for
clusters with even numbers of nodes - _especially_ 2-node
clusters. A single quorumd can serve dozens or hundreds
of clusters. Don't misunderstand - it's not
perfect, but it's reasonable.
Improbable to cause false positives for quorum.
You have to set up this quorum daemon, digital certificates
for client and server, and ideally put it in a cluster with
STONITH. So, 1-2 more servers, and quite a bit of
administrative overhead, and still the problem with
quorum for the quorum daemon cluster ;-).
Essential for clusters which are split across sites.
Most probable cause of its giving a false positive:
bug in the code.
ping-quorum-thing:
- A very simple poor-man's analog to the quorum daemon.
All you have to have is something both nodes in the
cluster can ping, and away you go. Very simple to
set up (just need something to ping), but certain
kinds of network failure conditions can result in false
positives. Here's what it takes to make this happen:
1. both nodes are really up but can't communicate
with each other by any communication path.
AND
2. Both nodes can ping this common node and
get a response back.
The most probable condition in which this will occur
is someone messing up the firewalls on _both_ machines
at the same time. Very odd switch failures can cause
this as well - but are very unlikely. Arp cache being
hosed might also cause this.
If the configuration includes
no ping nodes, then this module should return QUORUMTIE.
Very useful for a cluster which is _implementing_ the
Quorumd service - unless you have two in your site (meaning
four nodes).
two-node-quorum-tiebreaker:
This module just says "If I have 2 nodes, I have quorum".
This is very likely to produce false positive quorum
indications. Any time the two machines are both up and
can't communicate by any path, then this will ALWAYS
cause a false positive. It's literally trivial to set
up (it's the default). It returns QUORUMTIE if the
cluster has more than 2 nodes, or HASQUORUM otherwise.
And, what happens if your quorum service gives you a false positive?
In all cases the same thing: You might have both nodes
each STONITH the other - assuming the communication failure
still allows each node to communicate with the STONITH device.
Is data harmed? No.
Is it annyoing? Absolutely.
Will it loop until humans intervene? If rebooting the nodes
fixes the cluster communication, then it will stop. If it
doesn't, then it will repeat until something intervenes to
stop one of the nodes, or fix the network problem.
No method is perfect. Given enough perversity in the universe, any
method _can_ fail. Choose the most appropriate one for your environment.
Our current default is:
MajorityVote, TwoNode
In my view, the right _default_ configuration if we had all these
implemented is:
majorityvote
pingquorum
twonode
Then it would work like this:
If we have a majority then we have quorum
If we have exactly half the nodes, and have ping nodes
configured and we can ping all our ping nodes (or >n/2?)
then we have quorum
If we have exactly half the nodes and have no ping
nodes configured, and 2 nodes in
the configuration, then we have quorum.
You cannot make the quorum daemon the default. It is complex to
configure, and there is no way to automatically configure it in. So,
this is a better alternative for the default than the current
"majority,2node" configuration.
This does not diminish in any way Lars' comments about the quorum daemon
providing a higher quality result than this. That's by design.
But, this is a better two-node default than we could get any other way
that I know of. And, it is arguably a better default for any cluster
with an even number of nodes. One could easily give the pingquorum
module a parameter to tell it to return TIE if it's not dealing with a
2-node cluster.
It's worth repeating: data is never endangered if you're using STONITH.
--
Alan Robertson <[EMAIL PROTECTED]>
"Openness is the foundation and preservative of friendship... Let me
claim from you at all times your undisguised opinions." - William
Wilberforce
_______________________________________________________
Linux-HA-Dev: [email protected]
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
