On Mon, Jun 18, 2018 at 11:12 AM Jason Gauthier <jagauth...@gmail.com> wrote: > > On Mon, Jun 18, 2018 at 10:58 AM Ken Gaillot <kgail...@redhat.com> wrote: > > > > On Mon, 2018-06-18 at 10:10 -0400, Jason Gauthier wrote: > > > On Mon, Jun 18, 2018 at 9:55 AM Ken Gaillot <kgail...@redhat.com> > > > wrote: > > > > > > > > On Fri, 2018-06-15 at 21:39 -0400, Jason Gauthier wrote: > > > > > Greetings, > > > > > > > > > > Previously, I was using fiber channel with block devices. I > > > > > used > > > > > sbd to fence the disks, by creating a small block device, and > > > > > then > > > > > using stonith to fence the physical disk block. > > > > > > > > > > However, I had some reliability issues with that (I believe it > > > > > was > > > > > the > > > > > fibre channel interfacing, not clustering). So, I've moved > > > > > everything > > > > > to NFS. > > > > > > > > Another possibility would be to use qdevice with a third host, so > > > > that > > > > the cluster has true quorum, and then you can use sbd with hardware > > > > watchdog only (no need for a shared disk). > > > > > > This is the first time I've heard of qdevice. So, that said, the man > > > page indicates it's used on 2 node clusters, or even numbered? > > > > > > "It is recommended for clusters with an even number of nodes and > > > highly recommended for 2 node clusters." > > > > > > But that said, I have two nodes and a shared storage. I could make > > > the shared storage device a node, which seems a little odd. But > > > that > > > would make a 3rd. I do have another system, but the electricity costs > > > are already high for this project :) > > > > A qdevice host isn't a cluster node -- qdevice is a lightweight process > > that simply provides a quorum vote. > > > > I don't know what your shared storage device is, but if it's capable of > > running qdevice, I don't know of any problems with that (if you lose > > the storage, you can't do anything anyway, so it wouldn't be a big deal > > to lose quorum). It does sound like the shared storage device is a > > single point of failure. > > Yes, it is a single point of failure, for sure. I'm using NFS as the > shared storage platform. > I'll check out qdevice for all 3. > > > > > > My only resources are virtual machines running with KVM. So, I > > > > > am > > > > > > > > The VMs are resources, or nodes? The libvirt fence agents are for > > > > fencing VMs used as full cluster nodes. > > > > > > > > > > The VMs are resources. So, yeah, libvirt won't work. That > > > explanation helps. > > > It seems it would be simplest to protect each VM, so that if one node > > > went "missing" (but still running), the other node would not boot the > > > VMs and corrupt all the disks.
> > > That's what thought libvirt did. So, I still have to figure out > > > some > > > form of fencing for this. (Which you suggested above, but I have not > > > completely processed yet) > > > > Power fencing is almost always best. If that's not possible, you could > > try getting sbd working again. If that's not possible, and your shared > > storage device supports SCSI-3 persistent reservations, you could look > > into fence_scsi, which cuts off a node's access to the shared disk. Thanks for the help on this. I started looking at power fencing and then discovered that I can use IPMI. I've never used this before, so I wasn't sure. But sure enough. I set up two primitives, pointing to each other with a reboot action. Now, going to look into qnet/qdevice for the additional cluster vote. > The toss up was between iscsi and NFS, and I decided NFS was less hassle. > If I went iscsi, then I could create block devices and go back to an > sbd device as well. > > I just had a ton of problem with block devices not showing up at boot > when using fibre channel. > Perhaps the iscsi layer is completely different and that's a non-issue. > > > > > > trying to figure out what I should fence. I saw stonith has a > > > > > module, > > > > > external/libvirt, and that seems like it might work. But I can't > > > > > seem > > > > > to figure out how to use it with my crm config. > > > > > > > > > > I've attempted this: > > > > > primitive st_libvirt stonith:external/libvirt \ > > > > > params hypervisor_uri="qemu:///system" hostlist="alpha > > > > > beta" > > > > > \ > > > > > op monitor interval=2h \ > > > > > meta target-role=Stoppedprimitive st_libvirt > > > > > stonith:external/libvirt \ > > > > > params hypervisor_uri="qemu:///system" hostlist="alpha > > > > > beta" > > > > > \ > > > > > op monitor interval=2h > > > > > > > > > > But, I am not sure this is the correct syntax. The nodes are > > > > > alpha, > > > > > and beta. > > > > > > > > > > Any pointers appreciated. > > -- > > Ken Gaillot <kgail...@redhat.com> > > _______________________________________________ > > Users mailing list: Users@clusterlabs.org > > https://lists.clusterlabs.org/mailman/listinfo/users > > > > Project Home: http://www.clusterlabs.org > > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > > Bugs: http://bugs.clusterlabs.org _______________________________________________ Users mailing list: Users@clusterlabs.org https://lists.clusterlabs.org/mailman/listinfo/users Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org