I start to understand better how it works. I should have described my test infra and what are my goals, would have been more clear that what I've wrote. So, I have to nodes (physical machines) Both have 3 networks - ILo - Management (to access web interface) - Replication (for the mirroring, direct network cable between the 2 nodes) Vdisk are accessed throught Fibre channel by vmware infra
Here are the cases I wanna deal with: - 2 nodes are up / quadstor service up / vdisk OK but they can't communicate throught network (network or card issue) ---- I don't want to end up with neither of node fencing, refusing write -> storage down ---- I don't want to end up with 2 nodes fencing, both thinking they are masters, accepting writes request ending with data inconsistency - 1 node is down (no power, no ILo), the second can't fence and stop accepting writes request -> storage down Last case, more rare, but got to think about it: Let say I have 2 node N1 and N2 both have 2 storage pool P1 and P2 (I will have different pools, with differents type of disks and perf levels) and 2 Virtual Disk D1 (on pool P1) and D2 (on pool P2) in sync mirroring P1 is down on N1 and P2 is down on N2 (ie RAID failed) In that case, i can't fence by rebooting the other node Regarding that last case, when quadstor does the qmirrorcheck and fence, is it done for the one virtual disk / one storage pool or all virtual disks of the node? I will start working on scripts and will share them here, I guess I'm not the only one trying to replace high end SAN and need to reach same level of availability for the solution to be accepted. Le lundi 30 janvier 2017 14:54:57 UTC-5, quadstor a écrit : > > The criteria for fencing a peer node is if a command in a write > sequence has timed out. At this point to continue with the current > write command is to fence the peer node to prevent any further changes > w.r.t to the write command, and then it is assumed to be safe to > continue with the write command. > > Ofcourse it could be very well the case that the peer node is healthy > and the problem is in the network between the two. But even in that > case the only way to ensure continuity is fence the peer node and let > the current node handle the current and future writes. > > Given the above, there isn't a good way of doing below (the keep mirroring > part) > > "Ilo network is down or saying server is OFF / remote node services > are up (got a dedicated direct link for replication that could be used > to verify that)-> keep mirroring and don't takeover ownership" > > Mirroring can only be synchronous if the writes complete successfully > at both the ends > > The qmirrorcheck command does not necessarily have to be fence_ilo. It > can be a script which executes fence_ilo and evaluates various > conditions before determining a fence as successful or not. Also if > one fence (qmirrorcheck) command fails and if there are others > specified then they are also executed. The order is one listed by > qmirrorcheck. Maybe we should add an ordering here so that the fence > commands are always tried in the same order across reboots. Each fence > command is executed till one succeeds (an exit status of 0) If all > fence commands fails then the current writes and future writes are > failed till there is a reconnect to the peer node when the node or > network is up (The reconnect isn't automatic and occurs only on the > peer quadstor service startup) > > > On Tue, Jan 31, 2017 at 12:42 AM, Fabien Rouach <fabien...@gmail.com > <javascript:>> wrote: > > Thanks, > > Was easier than expected > > > > Now can i fence on several "criterias" and add some logic? > > I'd like to check if remote node Quadstor Service is running > > > > So i could evaluate cases like: > > Ilo network is down or saying server is OFF / remote node services are > up > > (got a dedicated direct link for replication that could be used to > verify > > that)-> keep mirroring and don't takeover ownership > > Ilo network return server is ON / remote node services are down -> > takeover > > ownership > > > > > > Le vendredi 27 janvier 2017 13:12:21 UTC-5, quadstor a écrit : > >> > >> The fence command usage for ILO isn't different from the other examples > >> > >> /quadstor/bin/qmirrorcheck -a -t fence -r 10.0.13.151 -v > >> '/usr/sbin/fence_ilo -a 10.0.13.151 -l Admin -p Password' > >> > >> Where 10.0.13.151 is the node we are trying to fence. You can test if > >> the fence command is communicating correctly with by sending '-o > >> status' to fence_ilo. (Note that its most likely > >> fence_ilo2/fence_ilo3/fence_ilo4) > >> > >> > >> On Wed, Jan 25, 2017 at 12:58 AM, Fabien Rouach <fabien...@gmail.com> > >> wrote: > >> > Hi, > >> > I'm actually building a test lab with Quadstor storage > virtualisation. > >> > Got 2 node (physicals machine in HA mirroring > >> > > >> > Would like to config fencing but all exemple i found used VM nodes, > and > >> > part > >> > of the setup is done on the VM host. > >> > Any guide / exemple available on how to do it with physical machine. > >> > > >> > I'd like to fence using network interface dedicated to replication. > >> > Optionally, I'like to fence using iLo interfaces (they are HP > machines), > >> > seen it's possible but seams complicated, anyone had already done > that? > >> > > >> > Is it possible to fence on 2 "criteria" (replication interface not > >> > beeing > >> > reachable + iLo ? > >> > > >> > Thanks > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups > >> > "QUADStor Storage Virtualization" group. > >> > To unsubscribe from this group and stop receiving emails from it, > send > >> > an > >> > email to quadstor-vir...@googlegroups.com. > >> > For more options, visit https://groups.google.com/d/optout. > > > > -- > > You received this message because you are subscribed to the Google > Groups > > "QUADStor Storage Virtualization" group. > > To unsubscribe from this group and stop receiving emails from it, send > an > > email to quadstor-vir...@googlegroups.com <javascript:>. > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "QUADStor Storage Virtualization" group. To unsubscribe from this group and stop receiving emails from it, send an email to quadstor-virt+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
