On Thu, May 03, 2007 at 09:26:56AM -0400, Doug Knight wrote: > Hmm, kill -9 on the active node is not sufficient to simulate a node > going down. Heartbeat goes away, but the file system remains mounted and > drbd remains primary on what was the active node.
Is there a way to force umount a drbd disk and 'kill' a drbd process (or was it a module; my drbd is very rusty)? > On Thu, 2007-05-03 at 09:08 -0400, Doug Knight wrote: > > > Thanks Dejan, I'll try the kill -9. One thing I'm seeing is that I can > > easily move the resources between nodes using the <location> constraint, > > but if I shutdown heartbeat on one node (/etc/init.d/heartbeat stop) I > > run into problems. If I shutdown the node with the active resources, > > heartbeat migrates the DRBD Master to the other node but the colocated > > group does not migrate (it remains stopped on the active node). I'm > > digging into that now. If I shutdown the node that does not have the > > active resources, the following happens: > > > > (State: DC on active node1, running drbd master and group resources) > > shutdown node2 > > demote attempted on node1 for drbd master, no attempt at halting groups > > resources that depend on drbd > > demote of drbd master fails due to "device held open" error, filesystem > > still has it mounted > > loops through continuously trying to demote drbd (spin condition) > > shutdown command never completes, control-C, then kill -9 main heartbeat > > on node1 > > drbd:0 goes stopped, :1 Master goes FAILED, group resources all still > > show started > > startup command executed on node1, Bad Things Happen, eventually drbd > > goes unmanaged > > after node1 heartbeat startup completes, stop group and drbd, restart > > resources, everything comes up fine > > > > I'm going to try a similar test, but using kill -9 right off the bat > > instead of the controlled shutdown. If there's any info I need to > > provide to make this clearer, please, anybody, just let me know. > > > > Doug > > > > On Thu, 2007-05-03 at 13:14 +0200, Dejan Muhamedagic wrote: > > > > > On Fri, Apr 27, 2007 at 03:10:22PM -0400, Doug Knight wrote: > > > > I now have a working configuration with DRBD master/slave, and a > > > > filesystem/pgsql/ipaddr group following it around. So far, I've been > > > > using a Place constraint and modifying its uname value to test the "fail > > > > over" of the resources. Can someone suggest a reasonable set of tests > > > > that most do to verify other possible error conditions (short of pulling > > > > the plug on one of the servers)? > > > > > > You can run CTS with your configuration. Otherwise, stopping > > > heartbeat in a way that it doesn't notice being stopped (kill -9) > > > simulates the "pull power plug" condition. You'd also want to > > > make various resources fail. > > > > > > > Also, the Place constraint is on the > > > > DRBD master/slave, does that make sense or should it be placed on one of > > > > the "higher level" resources like the file system or pgsql? > > > > > > I don't think it matters, you can go with either, given that the > > > resources are collocated. > > > > > > > Thanks, > > > > Doug > > > > > > > > On Thu, 2007-04-26 at 09:45 -0400, Doug Knight wrote: > > > > > > > > > Hi Alastair, > > > > > Have you encountered a situation where when you first start up the > > > > > drbd > > > > > master/slave resource, crm_mon and/or the GUI indicate Master status > > > > > on > > > > > one node, and Started status on the other (as opposed to Slave)? If > > > > > so, > > > > > how did you correct it? > > > > > > > > > > Doug > > > > > p.s. Thanks for the scripts and xml, they're a big help! > > > > > > > > > > > _______________________________________________ > > Linux-HA mailing list > > [email protected] > > http://lists.linux-ha.org/mailman/listinfo/linux-ha > > See also: http://linux-ha.org/ReportingProblems > > > _______________________________________________ > Linux-HA mailing list > [email protected] > http://lists.linux-ha.org/mailman/listinfo/linux-ha > See also: http://linux-ha.org/ReportingProblems -- Dejan _______________________________________________ Linux-HA mailing list [email protected] http://lists.linux-ha.org/mailman/listinfo/linux-ha See also: http://linux-ha.org/ReportingProblems
