15.03.2012 18:43, William Seligman wrote: > On 3/15/12 3:43 AM, Vladislav Bogdanov wrote: >> 14.03.2012 00:42, William Seligman wrote: >> [snip] >>> These were the log messages, which show that stonith_admin did its job and >>> CMAN >>> was notified of the fencing: <http://pastebin.com/jaH820Bv>. >> >> Could you please look at the output of 'dlm_tool ls' and 'dlm_tool dump'? >> >> You probably have 'kern_stop' and 'fencing' flags there. That means that >> dlm is unaware that node is fenced. > > Here's 'dlm_tool ls' with both nodes running cman+clvmd+gfs2: > <http://pastebin.com/QrZtm1Ue> > > 'dlm_tool dump': <http://pastebin.com/UKWxx9Y4> > > For comparison, I crashed one node and looked at the same output on the > remaining node: > dlm_tool ls: <http://pastebin.com/cKVAGxsd> > dlm_tool dump: <http://pastebin.com/c0h0p22Q> (the post-crash lines begin at > 1331824940)
Everything is fine there, dlm correctly understands that node is fenced and returns to a normal state. The only minor issue I see is that fencing took much time - 21 sec. > > I don't see the "kern_stop" or "fencing" flags. There's another thing I don't > see: at the top of 'dlm_tool dump' it displays most of the contents of my > cluster.conf file, except for the fencing sections. Here's my cluster.conf for > comparison: <http://pastebin.com/w5XNYyAX> It also looks correct (I mean fence_pcmk), but I can be wrong here, I do not use cman. > > cman doesn't see anything wrong in my cluster.conf file: > > # ccs_config_validate > Configuration validates > > But could there be something that's causing the fencing sections to be > ignored? > >>> >>> Unfortunately, I still got the gfs2 freeze, so this is not the complete >>> story. >> >> Both clvmd and gfs2 use dlm. If dlm layer thinks fencing is not >> completed, both of them freeze. > > I did 'grep -E "(dlm|clvm|fenc)" /var/log/messages' and looked at the time I > crashed the node: <http://pastebin.com/dvBtdLUs>. I see lines that indicate > that > pacemaker and drbd are fencing the node, but nothing from dlm or clvmd. Does > this indicate what you suggest: Could dlm somehow be ignoring or overlooking > the > fencing I put in? Is there any other way to check this? No, dlm_controld (and friends) mostly uses different logging method - that is what you see in dlm_tool dump. _______________________________________________ Linux-HA mailing list [email protected] http://lists.linux-ha.org/mailman/listinfo/linux-ha See also: http://linux-ha.org/ReportingProblems
