On 06/08/2011 02:44 PM, Klaus Darilion wrote: > > > Am 08.06.2011 19:04, schrieb Digimer: >> On 06/08/2011 12:50 PM, Klaus Darilion wrote: >>> I'm not sure if this question is better on corosync or pacemaker mailing >>> list - please advice if I'm wrong. >>> >>> >>> I have the following setup: >>> >>> Switch >>> / \ >>> / \ >>> eth0/ \eth0 >>> / \ >>> Server1<-------> Server2 >>> eth1 >>> >>> eth1 is used for DRBD replication. >>> >>> Now I want to use Pacemaker+Corosync to manage DRBD and the database >>> which uses the DRBD block device as data partition. The database is >>> accessed via an IP address in the eth0 network. >>> >>> I need to avoid a split-brain where DRBD becomes master on both server >>> and the database is started on. I experimented with corosync on eth0 or >>> eth1 or both (see other mail from today) but didn't find a proper >>> solution. >>> >>> I think I have to add other constraints to avoid split-brain, e.g. >>> pinging the default gateway. But pinging has a delay until the ping >>> primitive in pacemaker detects a failure. >>> >>> I think adding a 3rd node would also help as then I could use a quorum >>> to avoid split-brain. >>> >>> My questions: Where do I handle/avoid split-brain - on corosync layer or >>> pacemaker layer? >>> >>> Is there a best practice how to handle such scenarios? >>> >>> Shall I use corosync over eth0, eth1 or both (rrp)? >>> >>> If I use a 3rd node just for quorum - is a plain "corosync" node >>> sufficient or am I using also pacemaker with constraints to never run >>> the DRBD+database service on node3? >>> >>> Thanks >>> Klaus >> >> If you are concerned about split-brain in DRBD, you can put the >> protection into the DRBD config file. Look at: >> >> [...] > > > Hi Digimer! > > I fail to understand how fencing helps here. E.g. both network links of > node 2 are broken (service active on node1) > > node2 thinks that node1 is down, node2 wants to activate the service and > therefore kills node1 and then activates the service on node2. > > But this is not as it should be as node1 is still online and could serve > the clients. > > regards > Klaus
This is why I started with the node about DRBD. I am not familiar enough with Pacemaker to make suggestions there, but if/when Pacemaker is unable to prevent a split brain for whatever reason, the DRBD-level fencing will avoid the split brain. This works because at the moment a split-brain would have occurred, DRBD will instead block IO and call the fence script. Only when that script returns success (knowing reliably that the other node is dead), will it unblock IO. On the victim node, it should reboot as part of the fence call and, hopefully, recover in a healthier state. If the node still can't contact the surviving node, it should wait until the network issue is resolved before connecting and going online. Think of it like a last line of defence to avoid the split brain when all else has failed. -- Digimer E-Mail: [email protected] Freenode handle: digimer Papers and Projects: http://alteeve.com Node Assassin: http://nodeassassin.org "I feel confined, only free to expand myself within boundaries." _______________________________________________ Openais mailing list [email protected] https://lists.linux-foundation.org/mailman/listinfo/openais
