Hi, On Sun, Sep 11, 2016 at 6:28 PM, Gienek Nowacki <[email protected]> wrote:
> Hi, > > I have a problem with my cluster. > There are two nodes: wirt1v and wirt2v with pacemaker, corosync, dlm, > drbd (/dev/drbd2) and filesystem mounted as /virtfs2 with gfs2. > Each node contains LVM partition on wihich is DRBD. > > The situation is as follow: > > pcs cluster standby wirt2v > ...all is OK, there is possible to use /virtfs2 on wirt1v node > > pcs cluster unstandby wirt2v > This situation causes restarting/remounting Drbd2/Virtfs2 on the wirt1v > node. > Looks like you are missing the meta-attribute "interleave=true" for your filesystem clone: http://clusterlabs.org/doc/en-US/Pacemaker/1.1-pcs/html-single/Pacemaker_Explained/index.html#_clone_options Regards, Andreas > > And my question is as follow - why the resources restart on the wirt1v > node? > > I'm going to use /virtfs2 filesystem to install virtual machines, > so in such situation they will restart also. > > Couuld you advise me what to do with this problem? > > The logs and configs are below.. > > Thanks in advance, > Gienek Nowacki > ====================================== > #--------------------------------- > ### result: cat /etc/redhat-release ### > > CentOS Linux release 7.2.1511 (Core) > > #--------------------------------- > ### result: uname -a ### > > Linux wirt1v 3.10.0-327.28.3.el7.x86_64 #1 SMP Thu Aug 18 19:05:49 UTC > 2016 x86_64 x86_64 x86_64 GNU/Linux > > #--------------------------------- > ### result: cat /etc/hosts ### > > 127.0.0.1 localhost localhost.localdomain localhost4 > localhost4.localdomain4 > 172.31.0.23 wirt1 > 172.31.0.24 wirt2 > 1.1.1.1 wirt1v > 1.1.1.2 wirt2v > > #--------------------------------- > ### result: cat /etc/drbd.conf ### > > include "drbd.d/global_common.conf"; > include "drbd.d/*.res"; > > #--------------------------------- > ### result: cat /etc/drbd.d/global_common.conf ### > > common { > protocol C; > syncer { > verify-alg sha1; > } > net { > allow-two-primaries; > } > } > > #--------------------------------- > ### result: cat /etc/drbd.d/drbd2.res ### > > resource drbd2 { > meta-disk internal; > device /dev/drbd2; > on wirt1v { > disk /dev/vg02/drbd2; > address 1.1.1.1:7782; > } > on wirt2v { > disk /dev/vg02/drbd2; > address 1.1.1.2:7782; > } > } > #--------------------------------- > ### result: cat /proc/drbd ### > > version: 8.4.7-1 (api:1/proto:86-101) > GIT-hash: 3a6a769340ef93b1ba2792c6461250790795db49 build by phil@Build64R7, > 2016-01-12 14:29:40 > 2: cs:Connected ro:Primary/Primary ds:UpToDate/UpToDate C r----- > ns:132180 nr:44 dw:132224 dr:301948 al:33 bm:0 lo:0 pe:0 ua:0 ap:0 > ep:1 wo:f oos:0 > > #--------------------------------- > ### result: cat /etc/corosync/corosync.conf ### > > totem { > version: 2 > secauth: off > cluster_name: klasterek > transport: udpu > } > nodelist { > node { > ring0_addr: wirt1v > nodeid: 1 > } > node { > ring0_addr: wirt2v > nodeid: 2 > } > } > quorum { > provider: corosync_votequorum > two_node: 1 > } > logging { > to_logfile: yes > logfile: /var/log/cluster/corosync.log > to_syslog: yes > } > > #--------------------------------- > ### result: mnt ### > > #--------------------------------- > ### result: mount | grep virtfs2 ### > > /dev/drbd2 on /virtfs2 type gfs2 (rw,relatime,seclabel) > > > #--------------------------------- > ### result: pcs config ### > > Cluster Name: klasterek > Corosync Nodes: > wirt1v wirt2v > Pacemaker Nodes: > wirt1v wirt2v > Resources: > Clone: dlm-clone > Meta Attrs: clone-max=2 clone-node-max=1 > Resource: dlm (class=ocf provider=pacemaker type=controld) > Operations: start interval=0s timeout=90 (dlm-start-interval-0s) > stop interval=0s timeout=100 (dlm-stop-interval-0s) > monitor interval=60s (dlm-monitor-interval-60s) > Master: Drbd2-clone > Meta Attrs: master-max=2 master-node-max=1 clone-max=2 clone-node-max=1 > notify=true > Resource: Drbd2 (class=ocf provider=linbit type=drbd) > Attributes: drbd_resource=drbd2 > Operations: start interval=0s timeout=240 (Drbd2-start-interval-0s) > promote interval=0s timeout=90 (Drbd2-promote-interval-0s) > demote interval=0s timeout=90 (Drbd2-demote-interval-0s) > stop interval=0s timeout=100 (Drbd2-stop-interval-0s) > monitor interval=60s (Drbd2-monitor-interval-60s) > Clone: Virtfs2-clone > Resource: Virtfs2 (class=ocf provider=heartbeat type=Filesystem) > Attributes: device=/dev/drbd2 directory=/virtfs2 fstype=gfs2 > Operations: start interval=0s timeout=60 (Virtfs2-start-interval-0s) > stop interval=0s timeout=60 (Virtfs2-stop-interval-0s) > monitor interval=20 timeout=40 (Virtfs2-monitor-interval-20) > Stonith Devices: > Resource: fencing-idrac1 (class=stonith type=fence_idrac) > Attributes: pcmk_host_list=wirt1v ipaddr=172.31.0.223 lanplus=on > login=root passwd=my1secret2password3 > Operations: monitor interval=60 (fencing-idrac1-monitor-interval-60) > Resource: fencing-idrac2 (class=stonith type=fence_idrac) > Attributes: pcmk_host_list=wirt2v ipaddr=172.31.0.224 lanplus=on > login=root passwd=my1secret2password3 > Operations: monitor interval=60 (fencing-idrac2-monitor-interval-60) > Fencing Levels: > Location Constraints: > Ordering Constraints: > start dlm-clone then start Virtfs2-clone (kind:Mandatory) > (id:order-dlm-clone-Virtfs2-clone-mandatory) > promote Drbd2-clone then start Virtfs2-clone (kind:Mandatory) > (id:order-Drbd2-clone-Virtfs2-clone-mandatory) > Colocation Constraints: > Virtfs2-clone with Drbd2-clone (score:INFINITY) (with-rsc-role:Master) > (id:colocation-Virtfs2-clone-Drbd2-clone-INFINITY) > Virtfs2-clone with dlm-clone (score:INFINITY) > (id:colocation-Virtfs2-clone-dlm-clone-INFINITY) > Resources Defaults: > resource-stickiness: 100 > Operations Defaults: > No defaults set > Cluster Properties: > cluster-infrastructure: corosync > cluster-name: klasterek > dc-version: 1.1.13-10.el7_2.4-44eb2dd > have-watchdog: false > no-quorum-policy: ignore > stonith-enabled: true > symmetric-cluster: true > > #--------------------------------- > ### result: pcs status ### > > Cluster name: klasterek > Last updated: Sun Sep 11 16:02:15 2016 Last change: Sun Sep 11 > 15:01:05 2016 by root via crm_attribute on wirt2v > Stack: corosync > Current DC: wirt1v (version 1.1.13-10.el7_2.4-44eb2dd) - partition with > quorum > 2 nodes and 8 resources configured > Online: [ wirt1v wirt2v ] > Full list of resources: > fencing-idrac1 (stonith:fence_idrac): Started wirt1v > fencing-idrac2 (stonith:fence_idrac): Started wirt1v > Clone Set: dlm-clone [dlm] > Started: [ wirt1v wirt2v ] > Master/Slave Set: Drbd2-clone [Drbd2] > Masters: [ wirt1v wirt2v ] > Clone Set: Virtfs2-clone [Virtfs2] > Started: [ wirt1v wirt2v ] > PCSD Status: > wirt1v: Online > wirt2v: Online > Daemon Status: > corosync: active/disabled > pacemaker: active/disabled > pcsd: active/enabled > > #--------------------------------- > ### result: pcs property ### > > Cluster Properties: > cluster-infrastructure: corosync > cluster-name: klasterek > dc-version: 1.1.13-10.el7_2.4-44eb2dd > have-watchdog: false > no-quorum-policy: ignore > stonith-enabled: true > symmetric-cluster: true > > #--------------------------------- > ### result: pcs cluster cib ### > > <cib crm_feature_set="3.0.10" validate-with="pacemaker-2.3" epoch="105" > num_updates="11" admin_epoch="0" cib-last-written="Sun Sep 11 15:01:05 > 2016" update-origin="wirt2v" update-client="crm_attribute" > update-user="root" have-quorum="1" dc-uuid="1"> > <configuration> > <crm_config> > <cluster_property_set id="cib-bootstrap-options"> > <nvpair id="cib-bootstrap-options-have-watchdog" > name="have-watchdog" value="false"/> > <nvpair id="cib-bootstrap-options-dc-version" name="dc-version" > value="1.1.13-10.el7_2.4-44eb2dd"/> > <nvpair id="cib-bootstrap-options-cluster-infrastructure" > name="cluster-infrastructure" value="corosync"/> > <nvpair id="cib-bootstrap-options-cluster-name" > name="cluster-name" value="klasterek"/> > <nvpair id="cib-bootstrap-options-stonith-enabled" > name="stonith-enabled" value="true"/> > <nvpair id="cib-bootstrap-options-symmetric-cluster" > name="symmetric-cluster" value="true"/> > <nvpair id="cib-bootstrap-options-no-quorum-policy" > name="no-quorum-policy" value="ignore"/> > </cluster_property_set> > </crm_config> > <nodes> > <node id="1" uname="wirt1v"> > <instance_attributes id="nodes-1"/> > </node> > <node id="2" uname="wirt2v"> > <instance_attributes id="nodes-2"/> > </node> > </nodes> > <resources> > <primitive class="stonith" id="fencing-idrac1" type="fence_idrac"> > <instance_attributes id="fencing-idrac1-instance_attributes"> > <nvpair id="fencing-idrac1-instance_attributes-pcmk_host_list" > name="pcmk_host_list" value="wirt1v"/> > <nvpair id="fencing-idrac1-instance_attributes-ipaddr" > name="ipaddr" value="172.31.0.223"/> > <nvpair id="fencing-idrac1-instance_attributes-lanplus" > name="lanplus" value="on"/> > <nvpair id="fencing-idrac1-instance_attributes-login" > name="login" value="root"/> > <nvpair id="fencing-idrac1-instance_attributes-passwd" > name="passwd" value="my1secret2password3"/> > </instance_attributes> > <operations> > <op id="fencing-idrac1-monitor-interval-60" interval="60" > name="monitor"/> > </operations> > <meta_attributes id="fencing-idrac1-meta_attributes"/> > </primitive> > <primitive class="stonith" id="fencing-idrac2" type="fence_idrac"> > <instance_attributes id="fencing-idrac2-instance_attributes"> > <nvpair id="fencing-idrac2-instance_attributes-pcmk_host_list" > name="pcmk_host_list" value="wirt2v"/> > <nvpair id="fencing-idrac2-instance_attributes-ipaddr" > name="ipaddr" value="172.31.0.224"/> > <nvpair id="fencing-idrac2-instance_attributes-lanplus" > name="lanplus" value="on"/> > <nvpair id="fencing-idrac2-instance_attributes-login" > name="login" value="root"/> > <nvpair id="fencing-idrac2-instance_attributes-passwd" > name="passwd" value="my1secret2password3"/> > </instance_attributes> > <operations> > <op id="fencing-idrac2-monitor-interval-60" interval="60" > name="monitor"/> > </operations> > <meta_attributes id="fencing-idrac2-meta_attributes"/> > </primitive> > <clone id="dlm-clone"> > <primitive class="ocf" id="dlm" provider="pacemaker" > type="controld"> > <instance_attributes id="dlm-instance_attributes"/> > <operations> > <op id="dlm-start-interval-0s" interval="0s" name="start" > timeout="90"/> > <op id="dlm-stop-interval-0s" interval="0s" name="stop" > timeout="100"/> > <op id="dlm-monitor-interval-60s" interval="60s" > name="monitor"/> > </operations> > </primitive> > <meta_attributes id="dlm-clone-meta_attributes"> > <nvpair id="dlm-clone-max" name="clone-max" value="2"/> > <nvpair id="dlm-clone-node-max" name="clone-node-max" value="1"/> > </meta_attributes> > </clone> > <primitive class="ocf" id="Drbd2" provider="linbit" type="drbd"> > <instance_attributes id="Drbd2-instance_attributes"> > <nvpair id="Drbd2-instance_attributes-drbd_resource" > name="drbd_resource" value="drbd2"/> > </instance_attributes> > <operations> > <op id="Drbd2-start-interval-0s" interval="0s" name="start" > timeout="240"/> > <op id="Drbd2-promote-interval-0s" interval="0s" > name="promote" timeout="90"/> > <op id="Drbd2-demote-interval-0s" interval="0s" name="demote" > timeout="90"/> > <op id="Drbd2-stop-interval-0s" interval="0s" name="stop" > timeout="100"/> > <op id="Drbd2-monitor-interval-60s" interval="60s" > name="monitor"/> > </operations> > </primitive> > <meta_attributes id="Drbd2-clone-meta_attributes"> > <nvpair id="Drbd2-clone-meta_attributes-master-max" > name="master-max" value="2"/> > <nvpair id="Drbd2-clone-meta_attributes-master-node-max" > name="master-node-max" value="1"/> > <nvpair id="Drbd2-clone-meta_attributes-clone-max" > name="clone-max" value="2"/> > <nvpair id="Drbd2-clone-meta_attributes-clone-node-max" > name="clone-node-max" value="1"/> > <nvpair id="Drbd2-clone-meta_attributes-notify" name="notify" > value="true"/> > </meta_attributes> > </master> > <clone id="Virtfs2-clone"> > <primitive class="ocf" id="Virtfs2" provider="heartbeat" > type="Filesystem"> > <instance_attributes id="Virtfs2-instance_attributes"> > <nvpair id="Virtfs2-instance_attributes-device" name="device" > value="/dev/drbd2"/> > <nvpair id="Virtfs2-instance_attributes-directory" > name="directory" value="/virtfs2"/> > <nvpair id="Virtfs2-instance_attributes-fstype" name="fstype" > value="gfs2"/> > </instance_attributes> > <operations> > <op id="Virtfs2-start-interval-0s" interval="0s" name="start" > timeout="60"/> > <op id="Virtfs2-stop-interval-0s" interval="0s" name="stop" > timeout="60"/> > <op id="Virtfs2-monitor-interval-20" interval="20" > name="monitor" timeout="40"/> > </operations> > </primitive> > <meta_attributes id="Virtfs2-clone-meta_attributes"/> > </clone> > </resources> > <constraints> > <rsc_colocation id="colocation-Virtfs2-clone-Drbd2-clone-INFINITY" > rsc="Virtfs2-clone" score="INFINITY" with-rsc="Drbd2-clone" > with-rsc-role="Master"/> > <rsc_colocation id="colocation-Virtfs2-clone-dlm-clone-INFINITY" > rsc="Virtfs2-clone" score="INFINITY" with-rsc="dlm-clone"/> > <rsc_order first="dlm-clone" first-action="start" > id="order-dlm-clone-Virtfs2-clone-mandatory" then="Virtfs2-clone" > then-action="start"/> > <rsc_order first="Drbd2-clone" first-action="promote" > id="order-Drbd2-clone-Virtfs2-clone-mandatory" then="Virtfs2-clone" > then-action="start"/> > </constraints> > <rsc_defaults> > <meta_attributes id="rsc_defaults-options"> > <nvpair id="rsc_defaults-options-resource-stickiness" > name="resource-stickiness" value="100"/> > </meta_attributes> > </rsc_defaults> > </configuration> > <status> > <node_state id="1" uname="wirt1v" in_ccm="true" crmd="online" > crm-debug-origin="do_update_resource" join="member" expected="member"> > <transient_attributes id="1"> > <instance_attributes id="status-1"> > <nvpair id="status-1-shutdown" name="shutdown" value="0"/> > <nvpair id="status-1-probe_complete" name="probe_complete" > value="true"/> > <nvpair id="status-1-master-Drbd2" name="master-Drbd2" > value="10000"/> > </instance_attributes> > </transient_attributes> > <lrm id="1"> > <lrm_resources> > <lrm_resource id="Virtfs2" type="Filesystem" class="ocf" > provider="heartbeat"> > <lrm_rsc_op id="Virtfs2_last_0" operation_key="Virtfs2_start_0" > operation="start" crm-debug-origin="do_update_resource" > crm_feature_set="3.0.10" > transition-key="55:85:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > transition-magic="0:0;55:85:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > on_node="wirt1v" call-id="96" rc-code="0" op-status="0" interval="0" > last-run="1473598870" last-rc-change="1473598870" exec-time="639" > queue-time="1" op-digest="8dbd904c2115508ebcf3dffe8e7c6d82"/> > <lrm_rsc_op id="Virtfs2_monitor_20000" > operation_key="Virtfs2_monitor_20000" > operation="monitor" crm-debug-origin="do_update_resource" > crm_feature_set="3.0.10" > transition-key="56:85:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > transition-magic="0:0;56:85:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > on_node="wirt1v" call-id="97" rc-code="0" op-status="0" interval="20000" > last-rc-change="1473598871" exec-time="41" queue-time="0" op-digest=" > 051271837d1a8eccc0af38fbd8c406e4"/> > </lrm_resource> > <lrm_resource id="fencing-idrac1" type="fence_idrac" > class="stonith"> > <lrm_rsc_op id="fencing-idrac1_last_0" > operation_key="fencing-idrac1_start_0" operation="start" > crm-debug-origin="do_update_resource" crm_feature_set="3.0.10" > transition-key="9:58:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > transition-magic="0:0;9:58:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > on_node="wirt1v" call-id="58" rc-code="0" op-status="0" interval="0" > last-run="1473588685" last-rc-change="1473588685" exec-time="1045" > queue-time="0" op-digest="23a748cdf02f6f0fd03ac9823fc9bd52" > op-secure-params=" passwd " op-secure-digest=" > 2a5376722a6d891302b4e811e4de5c5a"/> > <lrm_rsc_op id="fencing-idrac1_monitor_60000" > operation_key="fencing-idrac1_monitor_60000" operation="monitor" > crm-debug-origin="do_update_resource" crm_feature_set="3.0.10" > transition-key="11:59:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > transition-magic="0:0;11:59:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > on_node="wirt1v" call-id="59" rc-code="0" op-status="0" interval="60000" > last-rc-change="1473588687" exec-time="84" queue-time="0" op-digest=" > 592f6bfb8f36e6645a6221de49f6f3b3" op-secure-params=" passwd " > op-secure-digest="2a5376722a6d891302b4e811e4de5c5a"/> > </lrm_resource> > <lrm_resource id="fencing-idrac2" type="fence_idrac" > class="stonith"> > <lrm_rsc_op id="fencing-idrac2_last_0" > operation_key="fencing-idrac2_start_0" operation="start" > crm-debug-origin="do_update_resource" crm_feature_set="3.0.10" > transition-key="14:62:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > transition-magic="0:0;14:62:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > on_node="wirt1v" call-id="61" rc-code="0" op-status="0" interval="0" > last-run="1473590528" last-rc-change="1473590528" exec-time="80" > queue-time="0" op-digest="268b7ef79bdf7a09609aa321d3d18a61" > op-secure-params=" passwd " op-secure-digest=" > f22e287dc4906f866a82eac0ab75d217"/> > <lrm_rsc_op id="fencing-idrac2_monitor_60000" > operation_key="fencing-idrac2_monitor_60000" operation="monitor" > crm-debug-origin="do_update_resource" crm_feature_set="3.0.10" > transition-key="15:62:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > transition-magic="0:0;15:62:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > on_node="wirt1v" call-id="62" rc-code="0" op-status="0" interval="60000" > last-rc-change="1473590529" exec-time="75" queue-time="1" op-digest=" > 40430ed0cd93e10fcba03a5e867b2af3" op-secure-params=" passwd " > op-secure-digest="f22e287dc4906f866a82eac0ab75d217"/> > </lrm_resource> > <lrm_resource id="dlm" type="controld" class="ocf" > provider="pacemaker"> > <lrm_rsc_op id="dlm_last_0" operation_key="dlm_start_0" > operation="start" crm-debug-origin="build_active_RAs" > crm_feature_set="3.0.10" > transition-key="14:15:0:29cf445c-f17d-4274-89e9-a869e4783c46" > transition-magic="0:0;14:15:0:29cf445c-f17d-4274-89e9-a869e4783c46" > on_node="wirt1v" call-id="25" rc-code="0" op-status="0" interval="0" > last-run="1473544487" last-rc-change="1473544487" exec-time="1082" > queue-time="0" op-digest="f2317cad3d54cec5d7d7aa7d0bf35cf8"/> > <lrm_rsc_op id="dlm_monitor_60000" > operation_key="dlm_monitor_60000" > operation="monitor" crm-debug-origin="build_active_RAs" > crm_feature_set="3.0.10" > transition-key="8:16:0:29cf445c-f17d-4274-89e9-a869e4783c46" > transition-magic="0:0;8:16:0:29cf445c-f17d-4274-89e9-a869e4783c46" > on_node="wirt1v" call-id="28" rc-code="0" op-status="0" interval="60000" > last-rc-change="1473544489" exec-time="38" queue-time="0" op-digest=" > 4811cef7f7f94e3a35a70be7916cb2fd"/> > </lrm_resource> > <lrm_resource id="Drbd2" type="drbd" class="ocf" > provider="linbit"> > <lrm_rsc_op id="Drbd2_last_0" operation_key="Drbd2_promote_0" > operation="promote" crm-debug-origin="build_active_RAs" > crm_feature_set="3.0.10" > transition-key="17:16:0:29cf445c-f17d-4274-89e9-a869e4783c46" > transition-magic="0:0;17:16:0:29cf445c-f17d-4274-89e9-a869e4783c46" > on_node="wirt1v" call-id="30" rc-code="0" op-status="0" interval="0" > last-run="1473544489" last-rc-change="1473544489" exec-time="58" > queue-time="0" op-digest="d0c8a735862843030d8426a5218ceb92"/> > </lrm_resource> > </lrm_resources> > </lrm> > </node_state> > <node_state id="2" uname="wirt2v" in_ccm="true" crmd="online" > crm-debug-origin="do_update_resource" join="member" expected="member"> > <transient_attributes id="2"> > <instance_attributes id="status-2"> > <nvpair id="status-2-shutdown" name="shutdown" value="0"/> > <nvpair id="status-2-probe_complete" name="probe_complete" > value="true"/> > <nvpair id="status-2-master-Drbd2" name="master-Drbd2" > value="10000"/> > </instance_attributes> > </transient_attributes> > <lrm id="2"> > <lrm_resources> > <lrm_resource id="fencing-idrac1" type="fence_idrac" > class="stonith"> > <lrm_rsc_op id="fencing-idrac1_last_0" > operation_key="fencing-idrac1_monitor_0" operation="monitor" > crm-debug-origin="do_update_resource" crm_feature_set="3.0.10" > transition-key="7:1:7:bfa57406-efca-4ca9-bdb5-01a121d172d8" > transition-magic="0:7;7:1:7:bfa57406-efca-4ca9-bdb5-01a121d172d8" > on_node="wirt2v" call-id="5" rc-code="7" op-status="0" interval="0" > last-run="1473544754" last-rc-change="1473544754" exec-time="1" > queue-time="0" op-digest="23a748cdf02f6f0fd03ac9823fc9bd52" > op-secure-params=" passwd " op-secure-digest=" > 2a5376722a6d891302b4e811e4de5c5a"/> > </lrm_resource> > <lrm_resource id="fencing-idrac2" type="fence_idrac" > class="stonith"> > <lrm_rsc_op id="fencing-idrac2_last_0" > operation_key="fencing-idrac2_stop_0" operation="stop" > crm-debug-origin="do_update_resource" crm_feature_set="3.0.10" > transition-key="13:62:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > transition-magic="0:0;13:62:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > on_node="wirt2v" call-id="55" rc-code="0" op-status="0" interval="0" > last-run="1473590528" last-rc-change="1473590528" exec-time="0" > queue-time="0" op-digest="268b7ef79bdf7a09609aa321d3d18a61" > op-secure-params=" passwd " op-secure-digest=" > f22e287dc4906f866a82eac0ab75d217"/> > <lrm_rsc_op id="fencing-idrac2_monitor_60000" > operation_key="fencing-idrac2_monitor_60000" operation="monitor" > crm-debug-origin="do_update_resource" crm_feature_set="3.0.10" > transition-key="13:59:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > transition-magic="0:0;13:59:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > on_node="wirt2v" call-id="53" rc-code="0" op-status="0" interval="60000" > last-rc-change="1473588689" exec-time="63" queue-time="0" op-digest=" > 40430ed0cd93e10fcba03a5e867b2af3" op-secure-params=" passwd " > op-secure-digest="f22e287dc4906f866a82eac0ab75d217"/> > </lrm_resource> > <lrm_resource id="dlm" type="controld" class="ocf" > provider="pacemaker"> > <lrm_rsc_op id="dlm_last_0" operation_key="dlm_start_0" > operation="start" crm-debug-origin="do_update_resource" > crm_feature_set="3.0.10" > transition-key="15:83:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > transition-magic="0:0;15:83:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > on_node="wirt2v" call-id="101" rc-code="0" op-status="0" interval="0" > last-run="1473598865" last-rc-change="1473598865" exec-time="1116" > queue-time="0" op-digest="f2317cad3d54cec5d7d7aa7d0bf35cf8"/> > <lrm_rsc_op id="dlm_monitor_60000" > operation_key="dlm_monitor_60000" > operation="monitor" crm-debug-origin="do_update_resource" > crm_feature_set="3.0.10" > transition-key="16:84:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > transition-magic="0:0;16:84:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > on_node="wirt2v" call-id="104" rc-code="0" op-status="0" interval="60000" > last-rc-change="1473598870" exec-time="47" queue-time="0" op-digest=" > 4811cef7f7f94e3a35a70be7916cb2fd"/> > </lrm_resource> > <lrm_resource id="Drbd2" type="drbd" class="ocf" > provider="linbit"> > <lrm_rsc_op id="Drbd2_last_0" operation_key="Drbd2_promote_0" > operation="promote" crm-debug-origin="do_update_resource" > crm_feature_set="3.0.10" > transition-key="27:84:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > transition-magic="0:0;27:84:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > on_node="wirt2v" call-id="106" rc-code="0" op-status="0" interval="0" > last-run="1473598870" last-rc-change="1473598870" exec-time="69" > queue-time="0" op-digest="d0c8a735862843030d8426a5218ceb92"/> > </lrm_resource> > <lrm_resource id="Virtfs2" type="Filesystem" class="ocf" > provider="heartbeat"> > <lrm_rsc_op id="Virtfs2_last_0" operation_key="Virtfs2_start_0" > operation="start" crm-debug-origin="do_update_resource" > crm_feature_set="3.0.10" > transition-key="53:85:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > transition-magic="0:0;53:85:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > on_node="wirt2v" call-id="108" rc-code="0" op-status="0" interval="0" > last-run="1473598870" last-rc-change="1473598870" exec-time="859" > queue-time="0" op-digest="8dbd904c2115508ebcf3dffe8e7c6d82"/> > <lrm_rsc_op id="Virtfs2_monitor_20000" > operation_key="Virtfs2_monitor_20000" > operation="monitor" crm-debug-origin="do_update_resource" > crm_feature_set="3.0.10" > transition-key="54:85:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > transition-magic="0:0;54:85:0:bfa57406-efca-4ca9-bdb5-01a121d172d8" > on_node="wirt2v" call-id="109" rc-code="0" op-status="0" interval="20000" > last-rc-change="1473598871" exec-time="50" queue-time="0" op-digest=" > 051271837d1a8eccc0af38fbd8c406e4"/> > </lrm_resource> > </lrm_resources> > </lrm> > </node_state> > </status> > </cib> > # ===================================================================== > # wirt1v-during_pcs-cluster-standby-wirt2v.log > # > Sep 11 14:58:06 wirt1v crmd[31951]: notice: State transition S_IDLE -> > S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL > origin=abort_transition_graph ] > Sep 11 14:58:06 wirt1v pengine[31950]: notice: On loss of CCM Quorum: > Ignore > Sep 11 14:58:06 wirt1v pengine[31950]: notice: Stop dlm:1#011(wirt2v) > Sep 11 14:58:06 wirt1v pengine[31950]: notice: Demote Drbd2:1#011(Master > -> Stopped wirt2v) > Sep 11 14:58:06 wirt1v pengine[31950]: notice: Stop > Virtfs2:1#011(wirt2v) > Sep 11 14:58:06 wirt1v pengine[31950]: notice: Calculated Transition 81: > /var/lib/pacemaker/pengine/pe-input-2852.bz2 > Sep 11 14:58:06 wirt1v crmd[31951]: notice: Initiating action 71: notify > Drbd2_pre_notify_demote_0 on wirt1v (local) > Sep 11 14:58:06 wirt1v crmd[31951]: notice: Initiating action 73: notify > Drbd2_pre_notify_demote_0 on wirt2v > Sep 11 14:58:06 wirt1v crmd[31951]: notice: Initiating action 54: stop > Virtfs2_stop_0 on wirt2v > Sep 11 14:58:06 wirt1v crmd[31951]: notice: Operation Drbd2_notify_0: ok > (node=wirt1v, call=86, rc=0, cib-update=0, confirmed=true) > Sep 11 14:58:06 wirt1v kernel: GFS2: fsid=klasterek:drbd2.0: recover > generation 2 done > Sep 11 14:58:06 wirt1v crmd[31951]: notice: Initiating action 17: stop > dlm_stop_0 on wirt2v > Sep 11 14:58:06 wirt1v crmd[31951]: notice: Initiating action 26: demote > Drbd2_demote_0 on wirt2v > Sep 11 14:58:06 wirt1v kernel: block drbd2: peer( Primary -> Secondary ) > Sep 11 14:58:06 wirt1v crmd[31951]: notice: Initiating action 72: notify > Drbd2_post_notify_demote_0 on wirt1v (local) > Sep 11 14:58:06 wirt1v crmd[31951]: notice: Initiating action 74: notify > Drbd2_post_notify_demote_0 on wirt2v > Sep 11 14:58:06 wirt1v crmd[31951]: notice: Operation Drbd2_notify_0: ok > (node=wirt1v, call=87, rc=0, cib-update=0, confirmed=true) > Sep 11 14:58:06 wirt1v crmd[31951]: notice: Initiating action 66: notify > Drbd2_pre_notify_stop_0 on wirt1v (local) > Sep 11 14:58:06 wirt1v crmd[31951]: notice: Initiating action 68: notify > Drbd2_pre_notify_stop_0 on wirt2v > Sep 11 14:58:06 wirt1v crmd[31951]: notice: Operation Drbd2_notify_0: ok > (node=wirt1v, call=88, rc=0, cib-update=0, confirmed=true) > Sep 11 14:58:06 wirt1v crmd[31951]: notice: Initiating action 27: stop > Drbd2_stop_0 on wirt2v > Sep 11 14:58:06 wirt1v kernel: drbd drbd2: peer( Secondary -> Unknown ) > conn( Connected -> TearDown ) pdsk( UpToDate -> DUnknown ) > Sep 11 14:58:06 wirt1v kernel: drbd drbd2: ack_receiver terminated > Sep 11 14:58:06 wirt1v kernel: drbd drbd2: Terminating drbd_a_drbd2 > Sep 11 14:58:06 wirt1v kernel: block drbd2: new current UUID > 12027A6DEC39CCB7:9A297D737BE3FBC7:214339307E5385FF:214239307E5385FF > Sep 11 14:58:06 wirt1v kernel: drbd drbd2: Connection closed > Sep 11 14:58:06 wirt1v kernel: drbd drbd2: conn( TearDown -> Unconnected ) > Sep 11 14:58:06 wirt1v kernel: drbd drbd2: receiver terminated > Sep 11 14:58:06 wirt1v kernel: drbd drbd2: Restarting receiver thread > Sep 11 14:58:06 wirt1v kernel: drbd drbd2: receiver (re)started > Sep 11 14:58:06 wirt1v kernel: drbd drbd2: conn( Unconnected -> > WFConnection ) > Sep 11 14:58:06 wirt1v crmd[31951]: warning: No match for shutdown action > on 2 > Sep 11 14:58:06 wirt1v crmd[31951]: notice: Transition aborted by > deletion of nvpair[@id='status-2-master-Drbd2']: Transient attribute > change (cib=0.104.3, source=abort_unless_down:333, > path=/cib/status/node_state[@id='2']/transient_attributes[@ > id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-master-Drbd2'], > 0) > Sep 11 14:58:06 wirt1v crmd[31951]: notice: Initiating action 67: notify > Drbd2_post_notify_stop_0 on wirt1v (local) > Sep 11 14:58:06 wirt1v crmd[31951]: notice: Operation Drbd2_notify_0: ok > (node=wirt1v, call=89, rc=0, cib-update=0, confirmed=true) > Sep 11 14:58:08 wirt1v crmd[31951]: notice: Transition 81 (Complete=28, > Pending=0, Fired=0, Skipped=0, Incomplete=0, > Source=/var/lib/pacemaker/pengine/pe-input-2852.bz2): > Complete > Sep 11 14:58:08 wirt1v pengine[31950]: notice: On loss of CCM Quorum: > Ignore > Sep 11 14:58:08 wirt1v pengine[31950]: notice: Calculated Transition 82: > /var/lib/pacemaker/pengine/pe-input-2853.bz2 > Sep 11 14:58:08 wirt1v crmd[31951]: notice: Transition 82 (Complete=0, > Pending=0, Fired=0, Skipped=0, Incomplete=0, > Source=/var/lib/pacemaker/pengine/pe-input-2853.bz2): > Complete > Sep 11 14:58:08 wirt1v crmd[31951]: notice: State transition > S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL > origin=notify_crmd ] > > # ===================================================================== > # wirt1v-during_pcs-cluster-unstandby-wirt2v.log > # > Sep 11 15:01:01 wirt1v systemd: Started Session 59 of user root. > Sep 11 15:01:01 wirt1v systemd: Starting Session 59 of user root. > Sep 11 15:01:05 wirt1v crmd[31951]: notice: State transition S_IDLE -> > S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL > origin=abort_transition_graph ] > Sep 11 15:01:05 wirt1v pengine[31950]: notice: On loss of CCM Quorum: > Ignore > Sep 11 15:01:05 wirt1v pengine[31950]: notice: Start dlm:1#011(wirt2v) > Sep 11 15:01:05 wirt1v pengine[31950]: notice: Start Drbd2:1#011(wirt2v) > Sep 11 15:01:05 wirt1v pengine[31950]: notice: Restart > Virtfs2:0#011(Started wirt1v) > Sep 11 15:01:05 wirt1v pengine[31950]: notice: Calculated Transition 83: > /var/lib/pacemaker/pengine/pe-input-2854.bz2 > Sep 11 15:01:05 wirt1v crmd[31951]: notice: Initiating action 15: start > dlm_start_0 on wirt2v > Sep 11 15:01:05 wirt1v crmd[31951]: notice: Initiating action 61: notify > Drbd2_pre_notify_start_0 on wirt1v (local) > Sep 11 15:01:05 wirt1v crmd[31951]: notice: Initiating action 51: stop > Virtfs2_stop_0 on wirt1v (local) > Sep 11 15:01:06 wirt1v Filesystem(Virtfs2)[42311]: INFO: Running stop for > /dev/drbd2 on /virtfs2 > Sep 11 15:01:06 wirt1v crmd[31951]: notice: Operation Drbd2_notify_0: ok > (node=wirt1v, call=90, rc=0, cib-update=0, confirmed=true) > Sep 11 15:01:06 wirt1v crmd[31951]: notice: Initiating action 25: start > Drbd2_start_0 on wirt2v > Sep 11 15:01:06 wirt1v Filesystem(Virtfs2)[42311]: INFO: Trying to unmount > /virtfs2 > Sep 11 15:01:06 wirt1v Filesystem(Virtfs2)[42311]: ERROR: Couldn't unmount > /virtfs2; trying cleanup with TERM > Sep 11 15:01:06 wirt1v crmd[31951]: notice: Transition aborted by > status-2-master-Drbd2, master-Drbd2=1000: Transient attribute change > (create cib=0.105.1, source=abort_unless_down:319, > path=/cib/status/node_state[@id='2']/transient_attributes[@ > id='2']/instance_attributes[@id='status-2'], 0) > Sep 11 15:01:06 wirt1v Filesystem(Virtfs2)[42311]: INFO: sending signal > TERM to: root 39937 39934 0 14:49 pts/0 Ss+ 0:00 -bash > Sep 11 15:01:06 wirt1v crmd[31951]: notice: Initiating action 62: notify > Drbd2_post_notify_start_0 on wirt1v (local) > Sep 11 15:01:06 wirt1v crmd[31951]: notice: Initiating action 63: notify > Drbd2_post_notify_start_0 on wirt2v > Sep 11 15:01:06 wirt1v crmd[31951]: notice: Operation Drbd2_notify_0: ok > (node=wirt1v, call=93, rc=0, cib-update=0, confirmed=true) > Sep 11 15:01:06 wirt1v kernel: drbd drbd2: Handshake successful: Agreed > network protocol version 101 > Sep 11 15:01:06 wirt1v kernel: drbd drbd2: Feature flags enabled on > protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME. > Sep 11 15:01:06 wirt1v kernel: drbd drbd2: conn( WFConnection -> > WFReportParams ) > Sep 11 15:01:06 wirt1v kernel: drbd drbd2: Starting ack_recv thread (from > drbd_r_drbd2 [32139]) > Sep 11 15:01:06 wirt1v kernel: block drbd2: drbd_sync_handshake: > Sep 11 15:01:06 wirt1v kernel: block drbd2: self 12027A6DEC39CCB7: > 9A297D737BE3FBC7:214339307E5385FF:214239307E5385FF bits:0 flags:0 > Sep 11 15:01:06 wirt1v kernel: block drbd2: peer 9A297D737BE3FBC6: > 0000000000000000:214339307E5385FE:214239307E5385FF bits:0 flags:0 > Sep 11 15:01:06 wirt1v kernel: block drbd2: uuid_compare()=1 by rule 70 > Sep 11 15:01:06 wirt1v kernel: block drbd2: peer( Unknown -> Secondary ) > conn( WFReportParams -> WFBitMapS ) pdsk( DUnknown -> Consistent ) > Sep 11 15:01:06 wirt1v kernel: block drbd2: send bitmap stats > [Bytes(packets)]: plain 0(0), RLE 23(1), total 23; compression: 100.0% > Sep 11 15:01:06 wirt1v kernel: block drbd2: receive bitmap stats > [Bytes(packets)]: plain 0(0), RLE 23(1), total 23; compression: 100.0% > Sep 11 15:01:06 wirt1v kernel: block drbd2: helper command: /sbin/drbdadm > before-resync-source minor-2 > Sep 11 15:01:06 wirt1v kernel: block drbd2: helper command: /sbin/drbdadm > before-resync-source minor-2 exit code 0 (0x0) > Sep 11 15:01:06 wirt1v kernel: block drbd2: conn( WFBitMapS -> SyncSource > ) pdsk( Consistent -> Inconsistent ) > Sep 11 15:01:06 wirt1v kernel: block drbd2: Began resync as SyncSource > (will sync 0 KB [0 bits set]). > Sep 11 15:01:06 wirt1v kernel: block drbd2: updated sync UUID > 12027A6DEC39CCB7:9A2A7D737BE3FBC7:9A297D737BE3FBC7:214339307E5385FF > Sep 11 15:01:06 wirt1v kernel: block drbd2: Resync done (total 1 sec; > paused 0 sec; 0 K/sec) > Sep 11 15:01:06 wirt1v kernel: block drbd2: updated UUIDs 12027A6DEC39CCB7: > 0000000000000000:9A2A7D737BE3FBC7:9A297D737BE3FBC7 > Sep 11 15:01:06 wirt1v kernel: block drbd2: conn( SyncSource -> Connected > ) pdsk( Inconsistent -> UpToDate ) > Sep 11 15:01:07 wirt1v Filesystem(Virtfs2)[42311]: ERROR: Couldn't unmount > /virtfs2; trying cleanup with TERM > Sep 11 15:01:07 wirt1v Filesystem(Virtfs2)[42311]: INFO: sending signal > TERM to: root 39937 39934 0 14:49 pts/0 Ss+ 0:00 -bash > Sep 11 15:01:08 wirt1v Filesystem(Virtfs2)[42311]: ERROR: Couldn't unmount > /virtfs2; trying cleanup with TERM > Sep 11 15:01:08 wirt1v Filesystem(Virtfs2)[42311]: INFO: sending signal > TERM to: root 39937 39934 0 14:49 pts/0 Ss+ 0:00 -bash > Sep 11 15:01:09 wirt1v Filesystem(Virtfs2)[42311]: ERROR: Couldn't unmount > /virtfs2; trying cleanup with KILL > Sep 11 15:01:09 wirt1v Filesystem(Virtfs2)[42311]: INFO: sending signal > KILL to: root 39937 39934 0 14:49 pts/0 Ss+ 0:00 -bash > Sep 11 15:01:09 wirt1v systemd-logind: Removed session 58. > Sep 11 15:01:10 wirt1v Filesystem(Virtfs2)[42311]: INFO: unmounted > /virtfs2 successfully > Sep 11 15:01:10 wirt1v lrmd[31948]: notice: Virtfs2_stop_0:42311:stderr [ > umount: /virtfs2: target is busy. ] > Sep 11 15:01:10 wirt1v lrmd[31948]: notice: Virtfs2_stop_0:42311:stderr > [ (In some cases useful info about processes that use ] > Sep 11 15:01:10 wirt1v lrmd[31948]: notice: Virtfs2_stop_0:42311:stderr > [ the device is found by lsof(8) or fuser(1)) ] > Sep 11 15:01:10 wirt1v lrmd[31948]: notice: Virtfs2_stop_0:42311:stderr [ > ocf-exit-reason:Couldn't unmount /virtfs2; trying cleanup with TERM ] > Sep 11 15:01:10 wirt1v lrmd[31948]: notice: Virtfs2_stop_0:42311:stderr [ > umount: /virtfs2: target is busy. ] > Sep 11 15:01:10 wirt1v lrmd[31948]: notice: Virtfs2_stop_0:42311:stderr > [ (In some cases useful info about processes that use ] > Sep 11 15:01:10 wirt1v lrmd[31948]: notice: Virtfs2_stop_0:42311:stderr > [ the device is found by lsof(8) or fuser(1)) ] > Sep 11 15:01:10 wirt1v lrmd[31948]: notice: Virtfs2_stop_0:42311:stderr [ > ocf-exit-reason:Couldn't unmount /virtfs2; trying cleanup with TERM ] > Sep 11 15:01:10 wirt1v lrmd[31948]: notice: Virtfs2_stop_0:42311:stderr [ > umount: /virtfs2: target is busy. ] > Sep 11 15:01:10 wirt1v lrmd[31948]: notice: Virtfs2_stop_0:42311:stderr > [ (In some cases useful info about processes that use ] > Sep 11 15:01:10 wirt1v lrmd[31948]: notice: Virtfs2_stop_0:42311:stderr > [ the device is found by lsof(8) or fuser(1)) ] > Sep 11 15:01:10 wirt1v lrmd[31948]: notice: Virtfs2_stop_0:42311:stderr [ > ocf-exit-reason:Couldn't unmount /virtfs2; trying cleanup with TERM ] > Sep 11 15:01:10 wirt1v lrmd[31948]: notice: Virtfs2_stop_0:42311:stderr [ > umount: /virtfs2: target is busy. ] > Sep 11 15:01:10 wirt1v lrmd[31948]: notice: Virtfs2_stop_0:42311:stderr > [ (In some cases useful info about processes that use ] > Sep 11 15:01:10 wirt1v lrmd[31948]: notice: Virtfs2_stop_0:42311:stderr > [ the device is found by lsof(8) or fuser(1)) ] > Sep 11 15:01:10 wirt1v lrmd[31948]: notice: Virtfs2_stop_0:42311:stderr [ > ocf-exit-reason:Couldn't unmount /virtfs2; trying cleanup with KILL ] > Sep 11 15:01:10 wirt1v crmd[31951]: notice: Operation Virtfs2_stop_0: ok > (node=wirt1v, call=92, rc=0, cib-update=178, confirmed=true) > Sep 11 15:01:10 wirt1v crmd[31951]: notice: Transition 83 (Complete=18, > Pending=0, Fired=0, Skipped=3, Incomplete=5, > Source=/var/lib/pacemaker/pengine/pe-input-2854.bz2): > Stopped > Sep 11 15:01:10 wirt1v pengine[31950]: notice: On loss of CCM Quorum: > Ignore > Sep 11 15:01:10 wirt1v pengine[31950]: notice: Promote Drbd2:1#011(Slave > -> Master wirt2v) > Sep 11 15:01:10 wirt1v pengine[31950]: notice: Start > Virtfs2:0#011(wirt1v) > Sep 11 15:01:10 wirt1v pengine[31950]: notice: Start > Virtfs2:1#011(wirt2v) > Sep 11 15:01:10 wirt1v pengine[31950]: notice: Calculated Transition 84: > /var/lib/pacemaker/pengine/pe-input-2855.bz2 > Sep 11 15:01:10 wirt1v crmd[31951]: notice: Initiating action 16: monitor > dlm_monitor_60000 on wirt2v > Sep 11 15:01:10 wirt1v crmd[31951]: notice: Initiating action 69: notify > Drbd2_pre_notify_promote_0 on wirt1v (local) > Sep 11 15:01:10 wirt1v crmd[31951]: notice: Initiating action 71: notify > Drbd2_pre_notify_promote_0 on wirt2v > Sep 11 15:01:10 wirt1v crmd[31951]: notice: Operation Drbd2_notify_0: ok > (node=wirt1v, call=94, rc=0, cib-update=0, confirmed=true) > Sep 11 15:01:10 wirt1v crmd[31951]: notice: Initiating action 27: promote > Drbd2_promote_0 on wirt2v > Sep 11 15:01:10 wirt1v kernel: block drbd2: peer( Secondary -> Primary ) > Sep 11 15:01:10 wirt1v crmd[31951]: notice: Initiating action 70: notify > Drbd2_post_notify_promote_0 on wirt1v (local) > Sep 11 15:01:10 wirt1v crmd[31951]: notice: Initiating action 72: notify > Drbd2_post_notify_promote_0 on wirt2v > Sep 11 15:01:10 wirt1v crmd[31951]: notice: Transition aborted by > status-2-master-Drbd2, master-Drbd2=10000: Transient attribute change > (modify cib=0.105.7, source=abort_unless_down:319, > path=/cib/status/node_state[@id='2']/transient_attributes[@ > id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-master-Drbd2'], > 0) > Sep 11 15:01:10 wirt1v crmd[31951]: notice: Operation Drbd2_notify_0: ok > (node=wirt1v, call=95, rc=0, cib-update=0, confirmed=true) > Sep 11 15:01:10 wirt1v crmd[31951]: notice: Transition 84 (Complete=13, > Pending=0, Fired=0, Skipped=2, Incomplete=5, > Source=/var/lib/pacemaker/pengine/pe-input-2855.bz2): > Stopped > Sep 11 15:01:10 wirt1v pengine[31950]: notice: On loss of CCM Quorum: > Ignore > Sep 11 15:01:10 wirt1v pengine[31950]: notice: Start > Virtfs2:0#011(wirt2v) > Sep 11 15:01:10 wirt1v pengine[31950]: notice: Start > Virtfs2:1#011(wirt1v) > Sep 11 15:01:10 wirt1v pengine[31950]: notice: Calculated Transition 85: > /var/lib/pacemaker/pengine/pe-input-2856.bz2 > Sep 11 15:01:10 wirt1v crmd[31951]: notice: Initiating action 53: start > Virtfs2_start_0 on wirt2v > Sep 11 15:01:10 wirt1v crmd[31951]: notice: Initiating action 55: start > Virtfs2:1_start_0 on wirt1v (local) > Sep 11 15:01:10 wirt1v Filesystem(Virtfs2)[42615]: INFO: Running start for > /dev/drbd2 on /virtfs2 > Sep 11 15:01:10 wirt1v kernel: GFS2: fsid=klasterek:drbd2: Trying to join > cluster "lock_dlm", "klasterek:drbd2" > Sep 11 15:01:10 wirt1v kernel: dlm: Using TCP for communications > Sep 11 15:01:10 wirt1v kernel: dlm: connecting to 2 > Sep 11 15:01:10 wirt1v kernel: dlm: got connection from 2 > Sep 11 15:01:11 wirt1v kernel: GFS2: fsid=klasterek:drbd2: first mounter > control generation 0 > Sep 11 15:01:11 wirt1v kernel: GFS2: fsid=klasterek:drbd2: Joined cluster. > Now mounting FS... > Sep 11 15:01:11 wirt1v kernel: GFS2: fsid=klasterek:drbd2.0: jid=0, > already locked for use > Sep 11 15:01:11 wirt1v kernel: GFS2: fsid=klasterek:drbd2.0: jid=0: > Looking at journal... > Sep 11 15:01:11 wirt1v kernel: GFS2: fsid=klasterek:drbd2.0: jid=0: Done > Sep 11 15:01:11 wirt1v kernel: GFS2: fsid=klasterek:drbd2.0: jid=1: Trying > to acquire journal lock... > Sep 11 15:01:11 wirt1v kernel: GFS2: fsid=klasterek:drbd2.0: jid=1: > Looking at journal... > Sep 11 15:01:11 wirt1v kernel: GFS2: fsid=klasterek:drbd2.0: jid=1: Done > Sep 11 15:01:11 wirt1v kernel: GFS2: fsid=klasterek:drbd2.0: first mount > done, others may mount > Sep 11 15:01:11 wirt1v crmd[31951]: notice: Operation Virtfs2_start_0: ok > (node=wirt1v, call=96, rc=0, cib-update=181, confirmed=true) > Sep 11 15:01:11 wirt1v crmd[31951]: notice: Initiating action 56: monitor > Virtfs2:1_monitor_20000 on wirt1v (local) > Sep 11 15:01:11 wirt1v crmd[31951]: notice: Initiating action 54: monitor > Virtfs2_monitor_20000 on wirt2v > Sep 11 15:01:11 wirt1v crmd[31951]: notice: Transition 85 (Complete=6, > Pending=0, Fired=0, Skipped=0, Incomplete=0, > Source=/var/lib/pacemaker/pengine/pe-input-2856.bz2): > Complete > Sep 11 15:01:11 wirt1v crmd[31951]: notice: State transition > S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL > origin=notify_crmd ] > Sep 11 15:01:18 wirt1v systemd-logind: New session 60 of user root. > Sep 11 15:01:18 wirt1v systemd: Started Session 60 of user root. > Sep 11 15:01:18 wirt1v systemd: Starting Session 60 of user root. > > # ===================================================================== > # wirt2v-during_pcs-cluster-standby-wirt2v.log > # > Sep 11 14:58:06 wirt2v crmd[27038]: notice: Operation Drbd2_notify_0: ok > (node=wirt2v, call=92, rc=0, cib-update=0, confirmed=true) > Sep 11 14:58:06 wirt2v Filesystem(Virtfs2)[28208]: INFO: Running stop for > /dev/drbd2 on /virtfs2 > Sep 11 14:58:06 wirt2v Filesystem(Virtfs2)[28208]: INFO: Trying to unmount > /virtfs2 > Sep 11 14:58:06 wirt2v Filesystem(Virtfs2)[28208]: INFO: unmounted > /virtfs2 successfully > Sep 11 14:58:06 wirt2v crmd[27038]: notice: Operation Virtfs2_stop_0: ok > (node=wirt2v, call=94, rc=0, cib-update=58, confirmed=true) > Sep 11 14:58:06 wirt2v kernel: dlm: closing connection to node 2 > Sep 11 14:58:06 wirt2v kernel: dlm: closing connection to node 1 > Sep 11 14:58:06 wirt2v kernel: block drbd2: role( Primary -> Secondary ) > Sep 11 14:58:06 wirt2v kernel: block drbd2: bitmap WRITE of 0 pages took 0 > jiffies > Sep 11 14:58:06 wirt2v kernel: block drbd2: 0 KB (0 bits) marked > out-of-sync by on disk bit-map. > Sep 11 14:58:06 wirt2v systemd-udevd: error: /dev/drbd2: Wrong medium type > Sep 11 14:58:06 wirt2v crmd[27038]: error: pcmkRegisterNode: Triggered > assert at xml.c:594 : node->type == XML_ELEMENT_NODE > Sep 11 14:58:06 wirt2v crmd[27038]: notice: Operation Drbd2_demote_0: ok > (node=wirt2v, call=97, rc=0, cib-update=59, confirmed=true) > Sep 11 14:58:06 wirt2v systemd-udevd: error: /dev/drbd2: Wrong medium type > Sep 11 14:58:06 wirt2v crmd[27038]: notice: Operation Drbd2_notify_0: ok > (node=wirt2v, call=98, rc=0, cib-update=0, confirmed=true) > Sep 11 14:58:06 wirt2v crmd[27038]: notice: Operation Drbd2_notify_0: ok > (node=wirt2v, call=99, rc=0, cib-update=0, confirmed=true) > Sep 11 14:58:06 wirt2v kernel: drbd drbd2: peer( Primary -> Unknown ) > conn( Connected -> Disconnecting ) pdsk( UpToDate -> DUnknown ) > Sep 11 14:58:06 wirt2v kernel: drbd drbd2: ack_receiver terminated > Sep 11 14:58:06 wirt2v kernel: drbd drbd2: Terminating drbd_a_drbd2 > Sep 11 14:58:06 wirt2v kernel: drbd drbd2: Connection closed > Sep 11 14:58:06 wirt2v kernel: drbd drbd2: conn( Disconnecting -> > StandAlone ) > Sep 11 14:58:06 wirt2v kernel: drbd drbd2: receiver terminated > Sep 11 14:58:06 wirt2v kernel: drbd drbd2: Terminating drbd_r_drbd2 > Sep 11 14:58:06 wirt2v kernel: block drbd2: disk( UpToDate -> Failed ) > Sep 11 14:58:06 wirt2v kernel: block drbd2: bitmap WRITE of 0 pages took 0 > jiffies > Sep 11 14:58:06 wirt2v kernel: block drbd2: 0 KB (0 bits) marked > out-of-sync by on disk bit-map. > Sep 11 14:58:06 wirt2v kernel: block drbd2: disk( Failed -> Diskless ) > Sep 11 14:58:06 wirt2v kernel: drbd drbd2: Terminating drbd_w_drbd2 > Sep 11 14:58:06 wirt2v crmd[27038]: notice: Operation Drbd2_stop_0: ok > (node=wirt2v, call=100, rc=0, cib-update=60, confirmed=true) > Sep 11 14:58:08 wirt2v crmd[27038]: notice: Operation dlm_stop_0: ok > (node=wirt2v, call=96, rc=0, cib-update=61, confirmed=true) > > # ===================================================================== > # wirt2v-during_pcs-cluster-unstandby-wirt2v.log > # > Sep 11 15:01:01 wirt2v systemd: Started Session 51 of user root. > Sep 11 15:01:01 wirt2v systemd: Starting Session 51 of user root. > Sep 11 15:01:06 wirt2v dlm_controld[28577]: 62718 dlm_controld 4.0.2 > started > Sep 11 15:01:06 wirt2v systemd-udevd: error: /dev/drbd2: Wrong medium type > Sep 11 15:01:06 wirt2v kernel: drbd drbd2: Starting worker thread (from > drbdsetup-84 [28610]) > Sep 11 15:01:06 wirt2v kernel: block drbd2: disk( Diskless -> Attaching ) > Sep 11 15:01:06 wirt2v kernel: drbd drbd2: Method to ensure write > ordering: flush > Sep 11 15:01:06 wirt2v kernel: block drbd2: max BIO size = 262144 > Sep 11 15:01:06 wirt2v kernel: block drbd2: drbd_bm_resize called with > capacity == 104854328 > Sep 11 15:01:06 wirt2v kernel: block drbd2: resync bitmap: bits=13106791 > words=204794 pages=400 > Sep 11 15:01:06 wirt2v kernel: block drbd2: size = 50 GB (52427164 KB) > Sep 11 15:01:06 wirt2v kernel: block drbd2: recounting of set bits took > additional 1 jiffies > Sep 11 15:01:06 wirt2v kernel: block drbd2: 0 KB (0 bits) marked > out-of-sync by on disk bit-map. > Sep 11 15:01:06 wirt2v kernel: block drbd2: disk( Attaching -> UpToDate ) > Sep 11 15:01:06 wirt2v kernel: block drbd2: attached to UUIDs > 9A297D737BE3FBC6:0000000000000000:214339307E5385FE:214239307E5385FF > Sep 11 15:01:06 wirt2v systemd-udevd: error: /dev/drbd2: Wrong medium type > Sep 11 15:01:06 wirt2v kernel: drbd drbd2: conn( StandAlone -> Unconnected > ) > Sep 11 15:01:06 wirt2v kernel: drbd drbd2: Starting receiver thread (from > drbd_w_drbd2 [28612]) > Sep 11 15:01:06 wirt2v kernel: drbd drbd2: receiver (re)started > Sep 11 15:01:06 wirt2v kernel: drbd drbd2: conn( Unconnected -> > WFConnection ) > Sep 11 15:01:06 wirt2v crmd[27038]: error: pcmkRegisterNode: Triggered > assert at xml.c:594 : node->type == XML_ELEMENT_NODE > Sep 11 15:01:06 wirt2v crmd[27038]: notice: Operation Drbd2_start_0: ok > (node=wirt2v, call=102, rc=0, cib-update=62, confirmed=true) > Sep 11 15:01:06 wirt2v crmd[27038]: notice: Operation Drbd2_notify_0: ok > (node=wirt2v, call=103, rc=0, cib-update=0, confirmed=true) > Sep 11 15:01:06 wirt2v kernel: drbd drbd2: Handshake successful: Agreed > network protocol version 101 > Sep 11 15:01:06 wirt2v kernel: drbd drbd2: Feature flags enabled on > protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME. > Sep 11 15:01:06 wirt2v kernel: drbd drbd2: conn( WFConnection -> > WFReportParams ) > Sep 11 15:01:06 wirt2v kernel: drbd drbd2: Starting ack_recv thread (from > drbd_r_drbd2 [28622]) > Sep 11 15:01:06 wirt2v kernel: block drbd2: drbd_sync_handshake: > Sep 11 15:01:06 wirt2v kernel: block drbd2: self 9A297D737BE3FBC6: > 0000000000000000:214339307E5385FE:214239307E5385FF bits:0 flags:0 > Sep 11 15:01:06 wirt2v kernel: block drbd2: peer 12027A6DEC39CCB7: > 9A297D737BE3FBC7:214339307E5385FF:214239307E5385FF bits:0 flags:0 > Sep 11 15:01:06 wirt2v kernel: block drbd2: uuid_compare()=-1 by rule 50 > Sep 11 15:01:06 wirt2v kernel: block drbd2: peer( Unknown -> Primary ) > conn( WFReportParams -> WFBitMapT ) disk( UpToDate -> Outdated ) pdsk( > DUnknown -> UpToDate ) > Sep 11 15:01:06 wirt2v kernel: block drbd2: receive bitmap stats > [Bytes(packets)]: plain 0(0), RLE 23(1), total 23; compression: 100.0% > Sep 11 15:01:06 wirt2v kernel: block drbd2: send bitmap stats > [Bytes(packets)]: plain 0(0), RLE 23(1), total 23; compression: 100.0% > Sep 11 15:01:06 wirt2v kernel: block drbd2: conn( WFBitMapT -> WFSyncUUID ) > Sep 11 15:01:06 wirt2v kernel: block drbd2: updated sync uuid > 9A2A7D737BE3FBC6:0000000000000000:214339307E5385FE:214239307E5385FF > Sep 11 15:01:06 wirt2v kernel: block drbd2: helper command: /sbin/drbdadm > before-resync-target minor-2 > Sep 11 15:01:06 wirt2v kernel: block drbd2: helper command: /sbin/drbdadm > before-resync-target minor-2 exit code 0 (0x0) > Sep 11 15:01:06 wirt2v kernel: block drbd2: conn( WFSyncUUID -> SyncTarget > ) disk( Outdated -> Inconsistent ) > Sep 11 15:01:06 wirt2v kernel: block drbd2: Began resync as SyncTarget > (will sync 0 KB [0 bits set]). > Sep 11 15:01:06 wirt2v kernel: block drbd2: Resync done (total 1 sec; > paused 0 sec; 0 K/sec) > Sep 11 15:01:06 wirt2v kernel: block drbd2: updated UUIDs 12027A6DEC39CCB6: > 0000000000000000:9A2A7D737BE3FBC6:9A297D737BE3FBC7 > Sep 11 15:01:06 wirt2v kernel: block drbd2: conn( SyncTarget -> Connected > ) disk( Inconsistent -> UpToDate ) > Sep 11 15:01:06 wirt2v kernel: block drbd2: helper command: /sbin/drbdadm > after-resync-target minor-2 > Sep 11 15:01:06 wirt2v kernel: block drbd2: helper command: /sbin/drbdadm > after-resync-target minor-2 exit code 0 (0x0) > Sep 11 15:01:07 wirt2v crmd[27038]: notice: Operation dlm_start_0: ok > (node=wirt2v, call=101, rc=0, cib-update=63, confirmed=true) > Sep 11 15:01:10 wirt2v crmd[27038]: notice: Operation Drbd2_notify_0: ok > (node=wirt2v, call=105, rc=0, cib-update=0, confirmed=true) > Sep 11 15:01:10 wirt2v kernel: block drbd2: role( Secondary -> Primary ) > Sep 11 15:01:10 wirt2v crmd[27038]: error: pcmkRegisterNode: Triggered > assert at xml.c:594 : node->type == XML_ELEMENT_NODE > Sep 11 15:01:10 wirt2v crmd[27038]: notice: Operation Drbd2_promote_0: ok > (node=wirt2v, call=106, rc=0, cib-update=65, confirmed=true) > Sep 11 15:01:10 wirt2v crmd[27038]: notice: Operation Drbd2_notify_0: ok > (node=wirt2v, call=107, rc=0, cib-update=0, confirmed=true) > Sep 11 15:01:10 wirt2v Filesystem(Virtfs2)[28772]: INFO: Running start for > /dev/drbd2 on /virtfs2 > Sep 11 15:01:10 wirt2v kernel: GFS2: fsid=klasterek:drbd2: Trying to join > cluster "lock_dlm", "klasterek:drbd2" > Sep 11 15:01:10 wirt2v kernel: dlm: Using TCP for communications > Sep 11 15:01:10 wirt2v kernel: dlm: connecting to 1 > Sep 11 15:01:10 wirt2v kernel: dlm: got connection from 1 > Sep 11 15:01:11 wirt2v kernel: GFS2: fsid=klasterek:drbd2: Joined cluster. > Now mounting FS... > Sep 11 15:01:11 wirt2v kernel: GFS2: fsid=klasterek:drbd2.1: jid=1, > already locked for use > Sep 11 15:01:11 wirt2v kernel: GFS2: fsid=klasterek:drbd2.1: jid=1: > Looking at journal... > Sep 11 15:01:11 wirt2v kernel: GFS2: fsid=klasterek:drbd2.1: jid=1: Done > Sep 11 15:01:11 wirt2v crmd[27038]: notice: Operation Virtfs2_start_0: ok > (node=wirt2v, call=108, rc=0, cib-update=66, confirmed=true) > > ============================================================== > > > > > > _______________________________________________ > Users mailing list: [email protected] > http://clusterlabs.org/mailman/listinfo/users > > Project Home: http://www.clusterlabs.org > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > Bugs: http://bugs.clusterlabs.org > >
_______________________________________________ Users mailing list: [email protected] http://clusterlabs.org/mailman/listinfo/users Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
