On 09/19/2016 02:31 AM, Auer, Jens wrote:
> Hi,
> 
> I am not sure that pacemaker should do any fencing here. In my setting, 
> corosync is configured to use a back-to-back connection for heartbeats. This 
> is different subnet then used by the ping resource that checks the network 
> connectivity and detects a failure. In my test, I bring down the network 
> device used by ping and this triggers the failover. The node status is known 
> by pacemaker since it receives heartbeats and it only a resource failure. I 
> asked for fencing conditions a few days ago, and basically was asserted that 
> resource failure should not trigger STONITH actions if not explicitly 
> configured.

Is the network interface being taken down here used for corosync
communication? If so, that is a node-level failure, and pacemaker will
fence.

There is a bit of a distinction between DRBD fencing and pacemaker
fencing. The DRBD configuration is designed so that DRBD's fencing
method is to go through pacemaker.

When DRBD is configured with 'fencing resource-only' and 'fence-peer
"/usr/lib/drbd/crm-fence-peer.sh";', and DRBD detects a network outage,
it will try to add a constraint that prevents the other node from
becoming master. It removes the constraint when connectivity is restored.

I am not familiar with all the under-the-hood details, but IIUC, if
pacemaker actually fences the node, then the other node can still take
over the DRBD. But if there is a network outage and no pacemaker
fencing, then you'll see the behavior you describe -- DRBD prevents
master takeover, to avoid stale data being used.


> I am also wondering why this is "sticky". After a failover test the DRBD 
> resources are not working even if I restart the cluster on all nodes. 
> 
> Best wishes,
>   Jens
> 
> --
> Dr. Jens Auer | CGI | Software Engineer
> CGI Deutschland Ltd. & Co. KG
> Rheinstraße 95 | 64295 Darmstadt | Germany
> T: +49 6151 36860 154
> jens.a...@cgi.com
> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter 
> de.cgi.com/pflichtangaben.
> 
> CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to CGI 
> Group Inc. and its affiliates may be contained in this message. If you are 
> not a recipient indicated or intended in this message (or responsible for 
> delivery of this message to such person), or you think for any reason that 
> this message may have been addressed to you in error, you may not use or copy 
> or deliver this message to anyone else. In such case, you should destroy this 
> message and are asked to notify the sender by reply e-mail.
> 
>> -----Original Message-----
>> From: Ken Gaillot [mailto:kgail...@redhat.com]
>> Sent: 16 September 2016 17:56
>> To: users@clusterlabs.org
>> Subject: Re: [ClusterLabs] No DRBD resource promoted to master in 
>> Active/Passive
>> setup
>>
>> On 09/16/2016 10:02 AM, Auer, Jens wrote:
>>> Hi,
>>>
>>> I have an Active/Passive configuration with a drbd mast/slave resource:
>>>
>>> MDA1PFP-S01 14:40:27 1803 0 ~ # pcs status Cluster name: MDA1PFP
>>> Last updated: Fri Sep 16 14:41:18 2016        Last change: Fri Sep 16
>>> 14:39:49 2016 by root via cibadmin on MDA1PFP-PCS01
>>> Stack: corosync
>>> Current DC: MDA1PFP-PCS02 (version 1.1.13-10.el7-44eb2dd) - partition
>>> with quorum
>>> 2 nodes and 7 resources configured
>>>
>>> Online: [ MDA1PFP-PCS01 MDA1PFP-PCS02 ]
>>>
>>> Full list of resources:
>>>
>>>  Master/Slave Set: drbd1_sync [drbd1]
>>>      Masters: [ MDA1PFP-PCS02 ]
>>>      Slaves: [ MDA1PFP-PCS01 ]
>>>  mda-ip    (ocf::heartbeat:IPaddr2):    Started MDA1PFP-PCS02
>>>  Clone Set: ping-clone [ping]
>>>      Started: [ MDA1PFP-PCS01 MDA1PFP-PCS02 ]
>>>  ACTIVE    (ocf::heartbeat:Dummy):    Started MDA1PFP-PCS02
>>>  shared_fs    (ocf::heartbeat:Filesystem):    Started MDA1PFP-PCS02
>>>
>>> PCSD Status:
>>>   MDA1PFP-PCS01: Online
>>>   MDA1PFP-PCS02: Online
>>>
>>> Daemon Status:
>>>   corosync: active/disabled
>>>   pacemaker: active/disabled
>>>   pcsd: active/enabled
>>>
>>> MDA1PFP-S01 14:41:19 1804 0 ~ # pcs resource --full
>>>  Master: drbd1_sync
>>>   Meta Attrs: master-max=1 master-node-max=1 clone-max=2
>>> clone-node-max=1 notify=true
>>>   Resource: drbd1 (class=ocf provider=linbit type=drbd)
>>>    Attributes: drbd_resource=shared_fs
>>>    Operations: start interval=0s timeout=240 (drbd1-start-interval-0s)
>>>                promote interval=0s timeout=90 (drbd1-promote-interval-0s)
>>>                demote interval=0s timeout=90 (drbd1-demote-interval-0s)
>>>                stop interval=0s timeout=100 (drbd1-stop-interval-0s)
>>>                monitor interval=60s (drbd1-monitor-interval-60s)
>>>  Resource: mda-ip (class=ocf provider=heartbeat type=IPaddr2)
>>>   Attributes: ip=192.168.120.20 cidr_netmask=32 nic=bond0
>>>   Operations: start interval=0s timeout=20s (mda-ip-start-interval-0s)
>>>               stop interval=0s timeout=20s (mda-ip-stop-interval-0s)
>>>               monitor interval=1s (mda-ip-monitor-interval-1s)
>>>  Clone: ping-clone
>>>   Resource: ping (class=ocf provider=pacemaker type=ping)
>>>    Attributes: dampen=5s multiplier=1000 host_list=pf-pep-dev-1
>>> timeout=1 attempts=3
>>>    Operations: start interval=0s timeout=60 (ping-start-interval-0s)
>>>                stop interval=0s timeout=20 (ping-stop-interval-0s)
>>>                monitor interval=1 (ping-monitor-interval-1)
>>>  Resource: ACTIVE (class=ocf provider=heartbeat type=Dummy)
>>>   Operations: start interval=0s timeout=20 (ACTIVE-start-interval-0s)
>>>               stop interval=0s timeout=20 (ACTIVE-stop-interval-0s)
>>>               monitor interval=10 timeout=20
>>> (ACTIVE-monitor-interval-10)
>>>  Resource: shared_fs (class=ocf provider=heartbeat type=Filesystem)
>>>   Attributes: device=/dev/drbd1 directory=/shared_fs fstype=xfs
>>>   Operations: start interval=0s timeout=60 (shared_fs-start-interval-0s)
>>>               stop interval=0s timeout=60 (shared_fs-stop-interval-0s)
>>>               monitor interval=20 timeout=40
>>> (shared_fs-monitor-interval-20)
>>>
>>> MDA1PFP-S01 14:41:35 1805 0 ~ # pcs constraint --full Location
>>> Constraints:
>>>   Resource: mda-ip
>>>     Enabled on: MDA1PFP-PCS01 (score:50)
>>> (id:location-mda-ip-MDA1PFP-PCS01-50)
>>>     Constraint: location-mda-ip
>>>       Rule: score=-INFINITY boolean-op=or  (id:location-mda-ip-rule)
>>>         Expression: pingd lt 1  (id:location-mda-ip-rule-expr)
>>>         Expression: not_defined pingd
>>> (id:location-mda-ip-rule-expr-1) Ordering Constraints:
>>>   start ping-clone then start mda-ip (kind:Optional)
>>> (id:order-ping-clone-mda-ip-Optional)
>>>   promote drbd1_sync then start shared_fs (kind:Mandatory)
>>> (id:order-drbd1_sync-shared_fs-mandatory)
>>> Colocation Constraints:
>>>   ACTIVE with mda-ip (score:INFINITY) (id:colocation-ACTIVE-mda-ip-INFINITY)
>>>   drbd1_sync with mda-ip (score:INFINITY) (rsc-role:Master)
>>> (with-rsc-role:Started) (id:colocation-drbd1_sync-mda-ip-INFINITY)
>>>   shared_fs with drbd1_sync (score:INFINITY) (rsc-role:Started)
>>> (with-rsc-role:Master) (id:colocation-shared_fs-drbd1_sync-INFINITY)
>>>
>>> The cluster starts fine, except resources starting not on the
>>> preferred host. I asked this in a different question to keep things 
>>> separated.
>>> The status after starting is:
>>> Last updated: Fri Sep 16 14:39:57 2016          Last change: Fri Sep 16
>>> 14:39:49 2016 by root via cibadmin on MDA1PFP-PCS01
>>> Stack: corosync
>>> Current DC: MDA1PFP-PCS02 (version 1.1.13-10.el7-44eb2dd) - partition
>>> with quorum
>>> 2 nodes and 7 resources configured
>>>
>>> Online: [ MDA1PFP-PCS01 MDA1PFP-PCS02 ]
>>>
>>>  Master/Slave Set: drbd1_sync [drbd1]
>>>      Masters: [ MDA1PFP-PCS02 ]
>>>      Slaves: [ MDA1PFP-PCS01 ]
>>> mda-ip  (ocf::heartbeat:IPaddr2):    Started MDA1PFP-PCS02
>>>  Clone Set: ping-clone [ping]
>>>      Started: [ MDA1PFP-PCS01 MDA1PFP-PCS02 ] ACTIVE
>>> (ocf::heartbeat:Dummy): Started MDA1PFP-PCS02
>>> shared_fs    (ocf::heartbeat:Filesystem):    Started MDA1PFP-PCS02
>>>
>>> From this state, I did two tests to simulate a cluster failover:
>>> 1. Shutdown the cluster node with the master with pcs cluster stop 2.
>>> Disable the network device for the virtual ip with ifdown and wait
>>> until ping detects it
>>>
>>> In both cases, the failover is executed but the drbd is not promoted
>>> to master on the new active node:
>>> Last updated: Fri Sep 16 14:43:33 2016          Last change: Fri Sep 16
>>> 14:43:31 2016 by root via cibadmin on MDA1PFP-PCS01
>>> Stack: corosync
>>> Current DC: MDA1PFP-PCS01 (version 1.1.13-10.el7-44eb2dd) - partition
>>> with quorum
>>> 2 nodes and 7 resources configured
>>>
>>> Online: [ MDA1PFP-PCS01 ]
>>> OFFLINE: [ MDA1PFP-PCS02 ]
>>>
>>>  Master/Slave Set: drbd1_sync [drbd1]
>>>      Slaves: [ MDA1PFP-PCS01 ]
>>> mda-ip  (ocf::heartbeat:IPaddr2):    Started MDA1PFP-PCS01
>>>  Clone Set: ping-clone [ping]
>>>      Started: [ MDA1PFP-PCS01 ]
>>> ACTIVE  (ocf::heartbeat:Dummy): Started MDA1PFP-PCS01
>>>
>>> I was able to trace this to the fencing in the drbd configuration
>>> MDA1PFP-S01 14:41:44 1806 0 ~ # cat /etc/drbd.d/shared_fs.res resource
>>> shared_fs {
>>> disk    /dev/mapper/rhel_mdaf--pf--pep--1-drbd;
>>>   disk {
>>>     fencing resource-only;
>>>   }
>>>   handlers {
>>>     fence-peer "/usr/lib/drbd/crm-fence-peer.sh";
>>>     after-resync-target "/usr/lib/drbd/crm-unfence-peer.sh";
>>>   }
>>>     device    /dev/drbd1;
>>>     meta-disk internal;
>>>     on MDA1PFP-S01 {
>>>         address 192.168.123.10:7789;
>>>     }
>>>     on MDA1PFP-S02 {
>>>         address 192.168.123.11:7789;
>>>     }
>>> }
>>
>> This coordinates fencing between DRBD and pacemaker. You still have to 
>> configure
>> fencing in pacemaker. If pacemaker can't fence the unseen node, it can't be 
>> sure it's
>> safe to bring up master.
>>
>>> I am using drbd 8.4.7, drbd utils 8.9.5 and pacemaker 2.3.4-7.el7 with
>>> corosyinc 0.9.143-15.el7 from the Centos7 repositories.
>>>
>>> MDA1PFP-S01 15:00:20 1841 0 ~ # drbdadm --version
>>> DRBDADM_BUILDTAG=GIT-hash:\
>> 5d50d9fb2a967d21c0f5746370ccc066d3a67f7d\
>>> build\ by\ mockbuild@\,\ 2016-01-12\ 12:46:45
>>> DRBDADM_API_VERSION=1
>>> DRBD_KERNEL_VERSION_CODE=0x080407
>>> DRBDADM_VERSION_CODE=0x080905
>>> DRBDADM_VERSION=8.9.5
>>>
>>> If I disable the fencing scripts everything works as expected. If
>>> enabled, no node is promoted to master after failover. It seems to be
>>> a sticky modificaton because once a failover is simulated with fencing
>>> scripts activated I cannot get the cluster to work anymore. Even
>>> removing the setting from the DRBD configuration does not help.
>>>
>>> I captured the complete log from /var/log/messages from cluster start
>>> to failover if that helps:
>>> MDA1PFP-S01 14:48:37 1807 0 ~ # cat /var/log/messages Sep 16 14:40:16
>>> MDA1PFP-S01 rsyslogd: [origin software="rsyslogd"
>>> swVersion="7.4.7" x-pid="13857" x-info="http://www.rsyslog.com";] start
>>> Sep 16 14:40:16 MDA1PFP-S01 rsyslogd-2221: module 'imuxsock' already
>>> in this config, cannot be added  [try http://www.rsyslog.com/e/2221 ]
>>> Sep 16 14:40:16 MDA1PFP-S01 systemd: Stopping System Logging Service...
>>> Sep 16 14:40:16 MDA1PFP-S01 systemd: Starting System Logging Service...
>>> Sep 16 14:40:16 MDA1PFP-S01 systemd: Started System Logging Service.
>>> Sep 16 14:40:27 MDA1PFP-S01 systemd: Started Corosync Cluster Engine.
>>> Sep 16 14:40:27 MDA1PFP-S01 systemd: Started Pacemaker High
>>> Availability Cluster Manager.
>>> Sep 16 14:43:30 MDA1PFP-S01 crmd[13130]:  notice: Operation
>>> ACTIVE_start_0: ok (node=MDA1PFP-PCS01, call=33, rc=0, cib-update=22,
>>> confirmed=true)
>>> Sep 16 14:43:30 MDA1PFP-S01 crmd[13130]:  notice: Operation
>>> drbd1_notify_0: ok (node=MDA1PFP-PCS01, call=32, rc=0, cib-update=0,
>>> confirmed=true)
>>> Sep 16 14:43:30 MDA1PFP-S01 IPaddr2(mda-ip)[15321]: INFO: Adding inet
>>> address 192.168.120.20/32 with broadcast address 192.168.120.255 to
>>> device bond0 Sep 16 14:43:30 MDA1PFP-S01 avahi-daemon[912]:
>>> Registering new address record for 192.168.120.20 on bond0.IPv4.
>>> Sep 16 14:43:30 MDA1PFP-S01 IPaddr2(mda-ip)[15321]: INFO: Bringing
>>> device bond0 up Sep 16 14:43:30 MDA1PFP-S01 kernel: block drbd1: peer(
>>> Primary -> Secondary ) Sep 16 14:43:30 MDA1PFP-S01
>>> IPaddr2(mda-ip)[15321]: INFO:
>>> /usr/libexec/heartbeat/send_arp -i 200 -r 5 -p
>>> /var/run/resource-agents/send_arp-192.168.120.20 bond0 192.168.120.20
>>> auto not_used not_used Sep 16 14:43:30 MDA1PFP-S01 crmd[13130]:
>>> notice: Operation
>>> mda-ip_start_0: ok (node=MDA1PFP-PCS01, call=35, rc=0, cib-update=24,
>>> confirmed=true)
>>> Sep 16 14:43:30 MDA1PFP-S01 crmd[13130]:  notice: Operation
>>> drbd1_notify_0: ok (node=MDA1PFP-PCS01, call=36, rc=0, cib-update=0,
>>> confirmed=true)
>>> Sep 16 14:43:30 MDA1PFP-S01 crmd[13130]:  notice: Operation
>>> drbd1_notify_0: ok (node=MDA1PFP-PCS01, call=38, rc=0, cib-update=0,
>>> confirmed=true)
>>> Sep 16 14:43:30 MDA1PFP-S01 kernel: drbd shared_fs: peer( Secondary ->
>>> Unknown ) conn( Connected -> TearDown ) pdsk( UpToDate -> DUnknown )
>>> Sep 16 14:43:30 MDA1PFP-S01 kernel: drbd shared_fs: ack_receiver
>>> terminated Sep 16 14:43:30 MDA1PFP-S01 kernel: drbd shared_fs:
>>> Terminating drbd_a_shared_f Sep 16 14:43:31 MDA1PFP-S01 kernel: drbd
>>> shared_fs: Connection closed Sep 16 14:43:31 MDA1PFP-S01 kernel: drbd
>>> shared_fs: conn( TearDown -> Unconnected ) Sep 16 14:43:31 MDA1PFP-S01
>>> kernel: drbd shared_fs: receiver terminated Sep 16 14:43:31
>>> MDA1PFP-S01 kernel: drbd shared_fs: Restarting receiver thread Sep 16
>>> 14:43:31 MDA1PFP-S01 kernel: drbd shared_fs: receiver (re)started Sep
>>> 16 14:43:31 MDA1PFP-S01 kernel: drbd shared_fs: conn( Unconnected ->
>>> WFConnection ) Sep 16 14:43:31 MDA1PFP-S01 crmd[13130]:  notice:
>>> Operation
>>> drbd1_notify_0: ok (node=MDA1PFP-PCS01, call=39, rc=0, cib-update=0,
>>> confirmed=true)
>>> Sep 16 14:43:31 MDA1PFP-S01 crmd[13130]:  notice: Operation
>>> drbd1_notify_0: ok (node=MDA1PFP-PCS01, call=40, rc=0, cib-update=0,
>>> confirmed=true)
>>> Sep 16 14:43:31 MDA1PFP-S01 kernel: drbd shared_fs: helper command:
>>> /sbin/drbdadm fence-peer shared_fs
>>> Sep 16 14:43:31 MDA1PFP-S01 crm-fence-peer.sh[15569]: invoked for
>>> shared_fs Sep 16 14:43:31 MDA1PFP-S01 crm-fence-peer.sh[15569]: INFO
>>> peer is not reachable, my disk is UpToDate: placed constraint
>>> 'drbd-fence-by-handler-shared_fs-drbd1_sync'
>>> Sep 16 14:43:31 MDA1PFP-S01 kernel: drbd shared_fs: helper command:
>>> /sbin/drbdadm fence-peer shared_fs exit code 5 (0x500) Sep 16 14:43:31
>>> MDA1PFP-S01 kernel: drbd shared_fs: fence-peer helper returned 5 (peer
>>> is unreachable, assumed to be dead) Sep 16 14:43:31 MDA1PFP-S01
>>> kernel: drbd shared_fs: pdsk( DUnknown -> Outdated ) Sep 16 14:43:31
>>> MDA1PFP-S01 kernel: block drbd1: role( Secondary -> Primary ) Sep 16
>>> 14:43:31 MDA1PFP-S01 kernel: block drbd1: new current UUID
>>>
>> B1FC3E9C008711DD:C02542C7B26F9B28:BCC6102B1FD69768:BCC5102B1FD697
>> 68
>>> Sep 16 14:43:31 MDA1PFP-S01 crmd[13130]:   error: pcmkRegisterNode:
>>> Triggered assert at xml.c:594 : node->type == XML_ELEMENT_NODE Sep 16
>>> 14:43:31 MDA1PFP-S01 crmd[13130]:  notice: Operation
>>> drbd1_promote_0: ok (node=MDA1PFP-PCS01, call=41, rc=0, cib-update=26,
>>> confirmed=true)
>>> Sep 16 14:43:31 MDA1PFP-S01 crmd[13130]:  notice: Operation
>>> drbd1_notify_0: ok (node=MDA1PFP-PCS01, call=42, rc=0, cib-update=0,
>>> confirmed=true)
>>> Sep 16 14:43:31 MDA1PFP-S01 crmd[13130]:  notice: Our peer on the DC
>>> (MDA1PFP-PCS02) is dead
>>> Sep 16 14:43:31 MDA1PFP-S01 crmd[13130]:  notice: State transition
>>> S_NOT_DC -> S_ELECTION [ input=I_ELECTION
>> cause=C_CRMD_STATUS_CALLBACK
>>> origin=peer_update_callback ] Sep 16 14:43:31 MDA1PFP-S01 crmd[13130]:
>>> notice: State transition S_ELECTION -> S_INTEGRATION [
>>> input=I_ELECTION_DC cause=C_TIMER_POPPED
>>> origin=election_timeout_popped ] Sep 16 14:43:31 MDA1PFP-S01
>>> attrd[13128]:  notice: crm_update_peer_proc:
>>> Node MDA1PFP-PCS02[2] - state is now lost (was member) Sep 16 14:43:31
>>> MDA1PFP-S01 attrd[13128]:  notice: Removing all
>>> MDA1PFP-PCS02 attributes for attrd_peer_change_cb Sep 16 14:43:31
>>> MDA1PFP-S01 attrd[13128]:  notice: Lost attribute writer
>>> MDA1PFP-PCS02
>>> Sep 16 14:43:31 MDA1PFP-S01 attrd[13128]:  notice: Removing
>>> MDA1PFP-PCS02/2 from the membership list Sep 16 14:43:31 MDA1PFP-S01
>>> attrd[13128]:  notice: Purged 1 peers with
>>> id=2 and/or uname=MDA1PFP-PCS02 from the membership cache Sep 16
>>> 14:43:31 MDA1PFP-S01 stonith-ng[13125]:  notice:
>>> crm_update_peer_proc: Node MDA1PFP-PCS02[2] - state is now lost (was
>>> member) Sep 16 14:43:31 MDA1PFP-S01 stonith-ng[13125]:  notice:
>>> Removing
>>> MDA1PFP-PCS02/2 from the membership list Sep 16 14:43:31 MDA1PFP-S01
>>> stonith-ng[13125]:  notice: Purged 1 peers with id=2 and/or
>>> uname=MDA1PFP-PCS02 from the membership cache Sep 16 14:43:31
>>> MDA1PFP-S01 cib[13124]:  notice: crm_update_peer_proc:
>>> Node MDA1PFP-PCS02[2] - state is now lost (was member) Sep 16 14:43:31
>>> MDA1PFP-S01 cib[13124]:  notice: Removing
>>> MDA1PFP-PCS02/2 from the membership list Sep 16 14:43:31 MDA1PFP-S01
>>> cib[13124]:  notice: Purged 1 peers with
>>> id=2 and/or uname=MDA1PFP-PCS02 from the membership cache Sep 16
>>> 14:43:31 MDA1PFP-S01 crmd[13130]: warning: FSA: Input I_ELECTION_DC
>>> from do_election_check() received in state S_INTEGRATION Sep 16
>>> 14:43:31 MDA1PFP-S01 crmd[13130]:  notice: Notifications disabled
>>> Sep 16 14:43:31 MDA1PFP-S01 crmd[13130]:   error: pcmkRegisterNode:
>>> Triggered assert at xml.c:594 : node->type == XML_ELEMENT_NODE Sep 16
>>> 14:43:31 MDA1PFP-S01 pengine[13129]:  notice: On loss of CCM
>>> Quorum: Ignore
>>> Sep 16 14:43:31 MDA1PFP-S01 pengine[13129]:  notice: Demote  drbd1:0
>>> (Master -> Slave MDA1PFP-PCS01)
>>> Sep 16 14:43:31 MDA1PFP-S01 pengine[13129]:  notice: Calculated
>>> Transition 0: /var/lib/pacemaker/pengine/pe-input-414.bz2
>>> Sep 16 14:43:31 MDA1PFP-S01 crmd[13130]:  notice: Initiating action 55:
>>> notify drbd1_pre_notify_demote_0 on MDA1PFP-PCS01 (local) Sep 16
>>> 14:43:31 MDA1PFP-S01 crmd[13130]:  notice: Operation
>>> drbd1_notify_0: ok (node=MDA1PFP-PCS01, call=43, rc=0, cib-update=0,
>>> confirmed=true)
>>> Sep 16 14:43:31 MDA1PFP-S01 crmd[13130]:  notice: Initiating action 8:
>>> demote drbd1_demote_0 on MDA1PFP-PCS01 (local) Sep 16 14:43:31
>>> MDA1PFP-S01 systemd-udevd: error: /dev/drbd1: Wrong medium type Sep 16
>>> 14:43:31 MDA1PFP-S01 kernel: block drbd1: role( Primary -> Secondary )
>>> Sep 16 14:43:31 MDA1PFP-S01 kernel: block drbd1: bitmap WRITE of 0
>>> pages took 0 jiffies Sep 16 14:43:31 MDA1PFP-S01 kernel: block drbd1:
>>> 0 KB (0 bits) marked out-of-sync by on disk bit-map.
>>> Sep 16 14:43:31 MDA1PFP-S01 systemd-udevd: error: /dev/drbd1: Wrong
>>> medium type
>>> Sep 16 14:43:31 MDA1PFP-S01 crmd[13130]:   error: pcmkRegisterNode:
>>> Triggered assert at xml.c:594 : node->type == XML_ELEMENT_NODE Sep 16
>>> 14:43:31 MDA1PFP-S01 crmd[13130]:  notice: Operation
>>> drbd1_demote_0: ok (node=MDA1PFP-PCS01, call=44, rc=0, cib-update=49,
>>> confirmed=true)
>>> Sep 16 14:43:31 MDA1PFP-S01 crmd[13130]:  notice: Initiating action 56:
>>> notify drbd1_post_notify_demote_0 on MDA1PFP-PCS01 (local) Sep 16
>>> 14:43:31 MDA1PFP-S01 crmd[13130]:  notice: Operation
>>> drbd1_notify_0: ok (node=MDA1PFP-PCS01, call=45, rc=0, cib-update=0,
>>> confirmed=true)
>>> Sep 16 14:43:31 MDA1PFP-S01 crmd[13130]:  notice: Initiating action 10:
>>> monitor drbd1_monitor_60000 on MDA1PFP-PCS01 (local) Sep 16 14:43:31
>>> MDA1PFP-S01 corosync[13019]: [TOTEM ] A new membership
>>> (192.168.121.10:988) was formed. Members left: 2 Sep 16 14:43:31
>>> MDA1PFP-S01 corosync[13019]: [QUORUM] Members[1]: 1 Sep 16 14:43:31
>>> MDA1PFP-S01 corosync[13019]: [MAIN  ] Completed service
>>> synchronization, ready to provide service.
>>> Sep 16 14:43:31 MDA1PFP-S01 pacemakerd[13113]:  notice:
>>> crm_reap_unseen_nodes: Node MDA1PFP-PCS02[2] - state is now lost (was
>>> member)
>>> Sep 16 14:43:31 MDA1PFP-S01 crmd[13130]:  notice: crm_reap_unseen_nodes:
>>> Node MDA1PFP-PCS02[2] - state is now lost (was member) Sep 16 14:43:31
>>> MDA1PFP-S01 crmd[13130]: warning: No match for shutdown action on 2
>>> Sep 16 14:43:31 MDA1PFP-S01 crmd[13130]:  notice: Stonith/shutdown of
>>> MDA1PFP-PCS02 not matched
>>> Sep 16 14:43:31 MDA1PFP-S01 crmd[13130]:  notice: Transition aborted:
>>> Node failure (source=peer_update_callback:252, 0)
>>> Sep 16 14:43:31 MDA1PFP-S01 crmd[13130]:   error: pcmkRegisterNode:
>>> Triggered assert at xml.c:594 : node->type == XML_ELEMENT_NODE Sep 16
>>> 14:43:31 MDA1PFP-S01 crmd[13130]:  notice: Transition 0 (Complete=10,
>>> Pending=0, Fired=0, Skipped=0, Incomplete=0,
>>> Source=/var/lib/pacemaker/pengine/pe-input-414.bz2): Complete Sep 16
>>> 14:43:31 MDA1PFP-S01 pengine[13129]:  notice: On loss of CCM
>>> Quorum: Ignore
>>> Sep 16 14:43:31 MDA1PFP-S01 pengine[13129]:  notice: Calculated
>>> Transition 1: /var/lib/pacemaker/pengine/pe-input-415.bz2
>>> Sep 16 14:43:31 MDA1PFP-S01 crmd[13130]:  notice: Transition 1
>>> (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
>>> Source=/var/lib/pacemaker/pengine/pe-input-415.bz2): Complete Sep 16
>>> 14:43:31 MDA1PFP-S01 crmd[13130]:  notice: State transition
>>> S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
>>> cause=C_FSA_INTERNAL origin=notify_crmd ] Sep 16 14:48:48 MDA1PFP-S01
>>> chronyd[909]: Source 62.116.162.126 replaced with 46.182.19.75
>>>
>>> Any help appreciated,
>>>   Jens
>>>
>>>
>>> --
>>> *Jens Auer *| CGI | Software-Engineer
>>> CGI (Germany) GmbH & Co. KG
>>> Rheinstraße 95 | 64295 Darmstadt | Germany
>>> T: +49 6151 36860 154
>>> _jens.auer@cgi.com_ <mailto:jens.a...@cgi.com> Unsere Pflichtangaben
>>> gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter
>>> _de.cgi.com/pflichtangaben_ <http://de.cgi.com/pflichtangaben>.
>>>
>>> CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging
>>> to CGI Group Inc. and its affiliates may be contained in this message.
>>> If you are not a recipient indicated or intended in this message (or
>>> responsible for delivery of this message to such person), or you think
>>> for any reason that this message may have been addressed to you in
>>> error, you may not use or copy or deliver this message to anyone else.
>>> In such case, you should destroy this message and are asked to notify
>>> the sender by reply e-mail.
>>>
>>> --
>>> *Jens Auer *| CGI | Software-Engineer
>>> CGI (Germany) GmbH & Co. KG
>>> Rheinstraße 95 | 64295 Darmstadt | Germany
>>> T: +49 6151 36860 154
>>> _jens.auer@cgi.com_ <mailto:jens.a...@cgi.com> Unsere Pflichtangaben
>>> gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter
>>> _de.cgi.com/pflichtangaben_ <http://de.cgi.com/pflichtangaben>.

_______________________________________________
Users mailing list: Users@clusterlabs.org
http://clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Reply via email to