OK, how about this: > Jul 22 14:03:41 zebrafish nfsserver(server_nfs)[6398]: INFO: Status: rpcbind > Jul 22 14:03:41 zebrafish nfsserver(server_nfs)[6398]: INFO: Status: > nfs-mountd > Jul 22 14:03:41 zebrafish nfsserver(server_nfs)[6398]: INFO: Status: > nfs-idmapd > Jul 22 14:03:41 zebrafish nfsserver(server_nfs)[6398]: INFO: Status: rpc-statd
...
> Jul 22 14:03:45 zebrafish crmd[1078]: notice: Transition aborted by
> status-1-master-drbd_storage doing create master-drbd_storage=1000: Transient
> attribute change
> Jul 22 14:03:45 zebrafish crmd[1078]: notice: Initiating notify operation
> drbd_storage_post_notify_start_0 locally on zebrafish
> Jul 22 14:03:45 zebrafish crmd[1078]: notice: Initiating notify operation
> drbd_storage:1_post_notify_start_0 on lionfish
> Jul 22 14:03:45 zebrafish crmd[1078]: notice: Result of notify operation for
> drbd_storage on zebrafish: 0 (ok)
> Jul 22 14:03:45 zebrafish crmd[1078]: notice: Transition 1 (Complete=21,
> Pending=0, Fired=0, Skipped=1, Incomplete=1,
> Source=/var/lib/pacemaker/pengine/pe-input-255.bz2): Stopped
> Jul 22 14:03:45 zebrafish pengine[1077]: notice: On loss of CCM Quorum:
> Ignore
> Jul 22 14:03:45 zebrafish pengine[1077]: notice: Demote
> drbd_storage:0#011(Master -> Slave zebrafish)
> Jul 22 14:03:45 zebrafish pengine[1077]: notice: Promote
> drbd_storage:1#011(Slave -> Master lionfish)
> Jul 22 14:03:45 zebrafish pengine[1077]: notice: Move
> drbd_filesystem#011(Started zebrafish -> lionfish)
> Jul 22 14:03:45 zebrafish pengine[1077]: notice: Move
> symlink_home#011(Started zebrafish -> lionfish)
> Jul 22 14:03:45 zebrafish pengine[1077]: notice: Move
> floating_ip#011(Started zebrafish -> lionfish)
> Jul 22 14:03:45 zebrafish pengine[1077]: notice: Move
> server_nfs#011(Started zebrafish -> lionfish)
> Jul 22 14:03:45 zebrafish pengine[1077]: notice: Move
> nfsshare_home#011(Started zebrafish -> lionfish)
> Jul 22 14:03:45 zebrafish pengine[1077]: notice: Move
> nfs_notify#011(Started zebrafish -> lionfish)
> Jul 22 14:03:45 zebrafish pengine[1077]: notice: Move
> symlink_etc_pki#011(Started zebrafish -> lionfish)
> Jul 22 14:03:45 zebrafish pengine[1077]: notice: Move
> symlink_etc_dovecot#011(Started zebrafish -> lionfish)
> Jul 22 14:03:45 zebrafish pengine[1077]: notice: Move
> symlink_var_dovecot#011(Started zebrafish -> lionfish)
> Jul 22 14:03:45 zebrafish pengine[1077]: notice: Move
> server_dovecot#011(Started zebrafish -> lionfish)
> Jul 22 14:03:45 zebrafish pengine[1077]: notice: Calculated transition 2,
> saving inputs in /var/lib/pacemaker/pengine/pe-input-256.bz2
> Jul 22 14:03:45 zebrafish crmd[1078]: notice: Initiating cancel operation
> drbd_storage_monitor_29000 locally on zebrafish
> Jul 22 14:03:45 zebrafish crmd[1078]: notice: Initiating stop operation
> nfs_notify_stop_0 locally on zebrafish
> Jul 22 14:03:45 zebrafish crmd[1078]: notice: Initiating stop operation
> server_dovecot_stop_0 locally on zebrafish
> Jul 22 14:03:45 zebrafish crmd[1078]: notice: Initiating notify operation
> drbd_storage_pre_notify_demote_0 locally on zebrafish
> Jul 22 14:03:45 zebrafish systemd: Reloading.
> Jul 22 14:03:45 zebrafish crmd[1078]: notice: Initiating notify operation
> drbd_storage_pre_notify_demote_0 on lionfish
> Jul 22 14:03:46 zebrafish nfsnotify(nfs_notify)[6511]: INFO: previous
> sm-notify processes terminated before stop action.
> Jul 22 14:03:46 zebrafish crmd[1078]: notice: Result of stop operation for
> nfs_notify on zebrafish: 0 (ok)
> Jul 22 14:03:46 zebrafish crmd[1078]: notice: Initiating stop operation
> nfsshare_home_stop_0 locally on zebrafish
> Jul 22 14:03:46 zebrafish crmd[1078]: notice: Result of notify operation for
> drbd_storage on zebrafish: 0 (ok)
> Jul 22 14:03:46 zebrafish systemd: Stopping Dovecot IMAP/POP3 email server...
> Jul 22 14:03:46 zebrafish systemd: Stopped Dovecot IMAP/POP3 email server.
> Jul 22 14:03:46 zebrafish exportfs(nfsshare_home)[6558]: INFO: Un-exporting
> file system ...
> Jul 22 14:03:46 zebrafish kernel: drbd raid: Handshake successful: Agreed
> network protocol version 101
> Jul 22 14:03:46 zebrafish kernel: drbd raid: Feature flags enabled on
> protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
> Jul 22 14:03:46 zebrafish kernel: drbd raid: conn( WFConnection ->
> WFReportParams )
> Jul 22 14:03:46 zebrafish kernel: drbd raid: Starting ack_recv thread (from
> drbd_r_raid [1626])
> Jul 22 14:03:46 zebrafish exportfs(nfsshare_home)[6558]: INFO: unexporting
> 144.92.167.128/25:/raid/home
> Jul 22 14:03:46 zebrafish exportfs(nfsshare_home)[6558]: INFO: Un-exported
> file system
> Jul 22 14:03:46 zebrafish crmd[1078]: notice: Result of stop operation for
> nfsshare_home on zebrafish: 0 (ok)
> Jul 22 14:03:46 zebrafish kernel: block drbd0: drbd_sync_handshake:
> Jul 22 14:03:46 zebrafish kernel: block drbd0: self
> F5C467DDAA8CAD23:BEA8E935E70AC8E8:30542C9D8D243C3C:30532C9D8D243C3D bits:9327
> flags:0
> Jul 22 14:03:46 zebrafish kernel: block drbd0: peer
> BEA8E935E70AC8E8:0000000000000000:30542C9D8D243C3D:30532C9D8D243C3D
> bits:1266688 flags:2
> Jul 22 14:03:46 zebrafish kernel: block drbd0: uuid_compare()=1 by rule 70
> Jul 22 14:03:46 zebrafish crmd[1078]: notice: Initiating stop operation
> symlink_home_stop_0 locally on zebrafish
> Jul 22 14:03:46 zebrafish crmd[1078]: notice: Initiating stop operation
> server_nfs_stop_0 locally on zebrafish
> Jul 22 14:03:46 zebrafish kernel: block drbd0: peer( Unknown -> Secondary )
> conn( WFReportParams -> WFBitMapS ) pdsk( DUnknown -> Consistent )
> Jul 22 14:03:46 zebrafish symlink(symlink_home)[6613]: INFO: removed '/home'
> Jul 22 14:03:46 zebrafish crmd[1078]: notice: Result of stop operation for
> symlin> k_home on zebrafish: 0 (ok)
> Jul 22 14:03:46 zebrafish kernel: block drbd0: drbd_sync_handshake:
> Jul 22 14:03:46 zebrafish kernel: block drbd0: self
> F5C467DDAA8CAD23:BEA8E935E70AC8E8:30542C9D8D243C3C:30532C9D8D243C3D bits:9327
> flags:0
> Jul 22 14:03:46 zebrafish kernel: block drbd0: peer
> BEA8E935E70AC8E8:0000000000000000:30542C9D8D243C3D:30532C9D8D243C3D
> bits:1266688 flags:2
> Jul 22 14:03:46 zebrafish kernel: block drbd0: uuid_compare()=1 by rule 70
> Jul 22 14:03:46 zebrafish crmd[1078]: notice: Initiating stop operation
> symlink_home_stop_0 locally on zebrafish
> Jul 22 14:03:46 zebrafish crmd[1078]: notice: Initiating stop operation
> server_nfs_stop_0 locally on zebrafish
> Jul 22 14:03:46 zebrafish kernel: block drbd0: peer( Unknown -> Secondary )
> conn( WFReportParams -> WFBitMapS ) pdsk( DUnknown -> Consistent )
> Jul 22 14:03:46 zebrafish symlink(symlink_home)[6613]: INFO: removed '/home'
> Jul 22 14:03:46 zebrafish crmd[1078]: notice: Result of stop operation for
> symlink_home on zebrafish: 0 (ok)
> Jul 22 14:03:46 zebrafish kernel: block drbd0: send bitmap stats
> [Bytes(packets)]: plain 0(0), RLE 1458(1), total 1458; compression: 100.0%
> Jul 22 14:03:46 zebrafish kernel: block drbd0: receive bitmap stats
> [Bytes(packets)]: plain 0(0), RLE 4700(2), total 4700; compression: 100.0%
> Jul 22 14:03:46 zebrafish kernel: block drbd0: helper command: /sbin/drbdadm
> before-resync-source minor-0
> Jul 22 14:03:46 zebrafish kernel: block drbd0: helper command: /sbin/drbdadm
> before-resync-source minor-0 exit code 0 (0x0)
> Jul 22 14:03:46 zebrafish kernel: block drbd0: conn( WFBitMapS -> SyncSource
> ) pdsk( Consistent -> Inconsistent )
> Jul 22 14:03:46 zebrafish kernel: block drbd0: Began resync as SyncSource
> (will sync 5084788 KB [1271197 bits set]).
> Jul 22 14:03:46 zebrafish kernel: block drbd0: updated sync UUID
> F5C467DDAA8CAD23:BEA9E935E70AC8E8:BEA8E935E70AC8E8:30542C9D8D243C3C
> Jul 22 14:03:46 zebrafish nfsserver(server_nfs)[6614]: INFO: Stopping NFS
> server ...
> Jul 22 14:03:46 zebrafish systemd: Stopping NFS server and services...
> Jul 22 14:03:46 zebrafish systemd: Stopped NFS server and services.
> Jul 22 14:03:46 zebrafish systemd: Stopping NFS Mount Daemon...
> Jul 22 14:03:46 zebrafish systemd: Stopping NFSv4 ID-name mapping service...
> Jul 22 14:03:46 zebrafish rpc.mountd[2655]: Caught signal 15, un-registering
> and exiting.
> Jul 22 14:03:46 zebrafish systemd: Stopped NFSv4 ID-name mapping service.
> Jul 22 14:03:46 zebrafish systemd: Stopped NFS Mount Daemon.
> Jul 22 14:03:46 zebrafish nfsserver(server_nfs)[6614]: INFO: Stop: threads
> Jul 22 14:03:46 zebrafish kernel: nfsd: last server has exited, flushing
> export cache
> Jul 22 14:03:46 zebrafish systemd: Stopping NFS status monitor for NFSv2/3
> locking....
> Jul 22 14:03:46 zebrafish systemd: Stopped NFS status monitor for NFSv2/3
> locking..
> Jul 22 14:03:46 zebrafish nfsserver(server_nfs)[6614]: INFO: Stop: rpc-statd
> Jul 22 14:03:46 zebrafish nfsserver(server_nfs)[6614]: INFO: Stop: nfs-idmapd
> Jul 22 14:03:46 zebrafish nfsserver(server_nfs)[6614]: INFO: Stop: nfs-mountd
> Jul 22 14:03:46 zebrafish systemd: Stopping RPC bind service...
> Jul 22 14:03:46 zebrafish systemd: Stopped RPC bind service.
> Jul 22 14:03:46 zebrafish nfsserver(server_nfs)[6614]: INFO: Stop: rpcbind
> Jul 22 14:03:46 zebrafish nfsserver(server_nfs)[6614]: INFO: Stop: rpc-gssd
> Jul 22 14:03:46 zebrafish nfsserver(server_nfs)[6614]: INFO: Stop: umount
> (1/10 attempts)
> Jul 22 14:03:47 zebrafish nfsserver(server_nfs)[6614]: INFO: NFS server
> stopped
> Jul 22 14:03:47 zebrafish crmd[1078]: notice: Result of stop operation for
> server_nfs on zebrafish: 0 (ok)
> Jul 22 14:03:47 zebrafish crmd[1078]: notice: Initiating stop operation
> floating_ip_stop_0 locally on zebrafish
> Jul 22 14:03:48 zebrafish crmd[1078]: notice: Result of stop operation for
> server_dovecot on zebrafish: 0 (ok)
> Jul 22 14:03:48 zebrafish crmd[1078]: notice: Initiating stop operation
> symlink_etc_pki_stop_0 locally on zebrafish
> Jul 22 14:03:48 zebrafish IPaddr2(floating_ip)[6769]: INFO: IP status = ok,
> IP_CIP=
> Jul 22 14:03:48 zebrafish crmd[1078]: notice: Initiating stop operation
> symlink_var_dovecot_stop_0 locally on zebrafish
> Jul 22 14:03:48 zebrafish crmd[1078]: notice: Result of stop operation for
> floating_ip on zebrafish: 0 (ok)
> Jul 22 14:03:48 zebrafish symlink(symlink_etc_pki)[6821]: INFO: removed
> '/etc/pki'
> Jul 22 14:03:48 zebrafish symlink(symlink_var_dovecot)[6822]: INFO: removed
> '/var/spool/dovecot'
> Jul 22 14:03:48 zebrafish crmd[1078]: notice: Result of stop operation for
> symlink_var_dovecot on zebrafish: 0 (ok)
> Jul 22 14:03:48 zebrafish crmd[1078]: notice: Initiating stop operation
> symlink_etc_dovecot_stop_0 locally on zebrafish
> Jul 22 14:03:48 zebrafish crmd[1078]: notice: Result of stop operation for
> symlink_etc_pki on zebrafish: 0 (ok)
> Jul 22 14:03:48 zebrafish symlink(symlink_etc_dovecot)[6863]: INFO: removed
> '/etc/dovecot'
> Jul 22 14:03:48 zebrafish crmd[1078]: notice: Result of stop operation for
> symlink_etc_dovecot on zebrafish: 0 (ok)
> Jul 22 14:03:48 zebrafish crmd[1078]: notice: Initiating stop operation
> drbd_filesystem_stop_0 locally on zebrafish
> Jul 22 14:03:48 zebrafish Filesystem(drbd_filesystem)[6886]: INFO: Running
> stop for /dev/drbd0 on /raid
> Jul 22 14:03:48 zebrafish Filesystem(drbd_filesystem)[6886]: INFO: Trying to
> unmount /raid
> Jul 22 14:03:48 zebrafish Filesystem(drbd_filesystem)[6886]: ERROR: Couldn't
> unmount /raid; trying cleanup with TERM
> Jul 22 14:03:48 zebrafish Filesystem(drbd_filesystem)[6886]: INFO: No
> processes on /raid were signalled. force_unmount is set to 'yes'
> Jul 22 14:03:49 zebrafish Filesystem(drbd_filesystem)[6886]: ERROR: Couldn't
> unmount /raid; trying cleanup with TERM
> Jul 22 14:03:49 zebrafish Filesystem(drbd_filesystem)[6886]: INFO: No
> processes on /raid were signalled. force_unmount is set to 'yes'
> Jul 22 14:03:50 zebrafish ntpd[596]: Deleting interface #8 enp2s0f0,
> 144.92.167.221#123, interface stats: received=0, sent=0, dropped=0,
> active_time=260 secs
> Jul 22 14:03:50 zebrafish Filesystem(drbd_filesystem)[6886]: ERROR: Couldn't
> unmount /raid; trying cleanup with TERM
> Jul 22 14:03:50 zebrafish Filesystem(drbd_filesystem)[6886]: INFO: No
> processes on /raid were signalled. force_unmount is set to 'yes'
> Jul 22 14:03:51 zebrafish Filesystem(drbd_filesystem)[6886]: ERROR: Couldn't
> unmount /raid; trying cleanup with KILL
> Jul 22 14:03:51 zebrafish Filesystem(drbd_filesystem)[6886]: INFO: No
> processes on /raid were signalled. force_unmount is set to 'yes'
> Jul 22 14:03:52 zebrafish Filesystem(drbd_filesystem)[6886]: ERROR: Couldn't
> unmount /raid; trying cleanup with KILL
> Jul 22 14:03:53 zebrafish Filesystem(drbd_filesystem)[6886]: INFO: No
> processes on /raid were signalled. force_unmount is set to 'yes'
> Jul 22 14:03:54 zebrafish Filesystem(drbd_filesystem)[6886]: ERROR: Couldn't
> unmount /raid; trying cleanup with KILL
> Jul 22 14:03:54 zebrafish Filesystem(drbd_filesystem)[6886]: INFO: No
> processes on /raid were signalled. force_unmount is set to 'yes'
> Jul 22 14:03:55 zebrafish Filesystem(drbd_filesystem)[6886]: ERROR: Couldn't
> unmount /raid, giving up!
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ umount: /raid: target is busy. ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ (In some cases useful info about
> processes that use ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ the device is found by lsof(8)
> or fuser(1)) ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ ocf-exit-reason:Couldn't unmount /raid;
> trying cleanup with TERM ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ umount: /raid: target is busy. ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ (In some cases useful info about
> processes that use ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ the device is found by lsof(8)
> or fuser(1)) ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ ocf-exit-reason:Couldn't unmount /raid;
> trying cleanup with TERM ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ umount: /raid: target is busy. ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ (In some cases useful info about
> processes that use ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ the device is found by lsof(8)
> or fuser(1)) ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ ocf-exit-reason:Couldn't unmount /raid;
> trying cleanup with TERM ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ umount: /raid: target is busy. ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ (In some cases useful info about
> processes that use ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ the device is found by lsof(8)
> or fuser(1)) ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ ocf-exit-reason:Couldn't unmount /raid;
> trying cleanup with KILL ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ umount: /raid: target is busy. ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ (In some cases useful info about
> processes that use ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ the device is found by lsof(8)
> or fuser(1)) ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ ocf-exit-reason:Couldn't unmount /raid;
> trying cleanup with KILL ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ umount: /raid: target is busy. ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ (In some cases useful info about
> processes that use ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ the device is found by lsof(8)
> or fuser(1)) ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ ocf-exit-reason:Couldn't unmount /raid;
> trying cleanup with KILL ]
> Jul 22 14:03:55 zebrafish lrmd[1075]: notice:
> drbd_filesystem_stop_0:6886:stderr [ ocf-exit-reason:Couldn't unmount /raid,
> giving up! ]
> Jul 22 14:03:55 zebrafish crmd[1078]: notice: Result of stop operation for
> drbd_filesystem on zebrafish: 1 (unknown error)
> Jul 22 14:03:55 zebrafish crmd[1078]: notice:
> zebrafish-drbd_filesystem_stop_0:101 [ umount: /raid: target is busy.\n
> (In some cases useful info about processes that use\n the device is
> found by lsof(8) or fuser(1))\nocf-exit-reason:Couldn't unmount /raid; trying
> cleanup with TERM\numount: /raid: target is busy.\n (In some cases
> useful info about processes that use\n the device is found by lsof(8)
> or fuser(1))\nocf-exit-reason:Couldn't unmount /raid; trying cleanup with
> TERM\numount: /raid: target is busy.\n
> Jul 22 14:03:55 zebrafish crmd[1078]: warning: Action 45
> (drbd_filesystem_stop_0) on zebrafish failed (target: 0 vs. rc: 1): Error
> Jul 22 14:03:55 zebrafish crmd[1078]: notice: Transition aborted by
> operation drbd_filesystem_stop_0 'modify' on zebrafish: Event failed
> Jul 22 14:03:55 zebrafish crmd[1078]: warning: Action 45
> (drbd_filesystem_stop_0) on zebrafish failed (target: 0 vs. rc: 1): Error
> Jul 22 14:03:55 zebrafish crmd[1078]: notice: Transition 2 (Complete=21,
> Pending=0, Fired=0, Skipped=0, Incomplete=43,
> Source=/var/lib/pacemaker/pengine/pe-input-256.bz2): Complete
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: On loss of CCM Quorum:
> Ignore
> Jul 22 14:03:55 zebrafish pengine[1077]: warning: Processing failed op stop
> for drbd_filesystem on zebrafish: unknown error (1)
> Jul 22 14:03:55 zebrafish pengine[1077]: warning: Processing failed op stop
> for drbd_filesystem on zebrafish: unknown error (1)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Demote
> drbd_storage:0#011(Master -> Slave zebrafish)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Promote
> drbd_storage:1#011(Slave -> Master lionfish)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Start
> floating_ip#011(lionfish - blocked)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Start
> server_nfs#011(lionfish - blocked)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Start
> nfsshare_home#011(lionfish - blocked)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Start
> nfs_notify#011(lionfish - blocked)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Start
> symlink_etc_pki#011(lionfish - blocked)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Start
> symlink_etc_dovecot#011(lionfish - blocked)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Start
> symlink_var_dovecot#011(lionfish - blocked)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Start
> server_dovecot#011(lionfish - blocked)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Calculated transition 3,
> saving inputs in /var/lib/pacemaker/pengine/pe-input-257.bz2
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: On loss of CCM Quorum:
> Ignore
> Jul 22 14:03:55 zebrafish pengine[1077]: warning: Processing failed op stop
> for drbd_filesystem on zebrafish: unknown error (1)
> Jul 22 14:03:55 zebrafish pengine[1077]: warning: Processing failed op stop
> for drbd_filesystem on zebrafish: unknown error (1)
> Jul 22 14:03:55 zebrafish pengine[1077]: warning: Forcing drbd_filesystem
> away from zebrafish after 1000000 failures (max=1000000)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Demote
> drbd_storage:0#011(Master -> Slave zebrafish)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Promote
> drbd_storage:1#011(Slave -> Master lionfish)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Start
> floating_ip#011(lionfish - blocked)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Start
> server_nfs#011(lionfish - blocked)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Start
> nfsshare_home#011(lionfish - blocked)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Start
> nfs_notify#011(lionfish - blocked)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Start
> symlink_etc_pki#011(lionfish - blocked)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Start
> symlink_etc_dovecot#011(lionfish - blocked)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Start
> symlink_var_dovecot#011(lionfish - blocked)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Start
> server_dovecot#011(lionfish - blocked)
> Jul 22 14:03:55 zebrafish pengine[1077]: notice: Calculated transition 4,
> saving inputs in /var/lib/pacemaker/pengine/pe-input-258.bz2
> Jul 22 14:03:55 zebrafish crmd[1078]: notice: Initiating demote operation
> drbd_storage_demote_0 locally on zebrafish
> Jul 22 14:03:55 zebrafish crmd[1078]: notice: Initiating notify operation
> drbd_storage_pre_notify_demote_0 locally on zebrafish
> Jul 22 14:03:55 zebrafish crmd[1078]: notice: Initiating notify operation
> drbd_storage_pre_notify_demote_0 on lionfish
> Jul 22 14:03:56 zebrafish kernel: block drbd0: State change failed: Device is
> held open by someone
> Jul 22 14:03:56 zebrafish kernel: block drbd0: state = { cs:SyncSource
> ro:Primary/Secondary ds:UpToDate/Inconsistent r----- }
> Jul 22 14:03:56 zebrafish kernel: block drbd0: wanted = { cs:SyncSource
> ro:Secondary/Secondary ds:UpToDate/Inconsistent r----- }
> Jul 22 14:03:56 zebrafish drbd(drbd_storage)[7093]: ERROR: raid: Called
> drbdadm -c /etc/drbd.conf secondary raid
> Jul 22 14:03:56 zebrafish drbd(drbd_storage)[7093]: ERROR: raid: Exit code 11
> Jul 22 14:03:56 zebrafish drbd(drbd_storage)[7093]: ERROR: raid: Command
> output:
> Jul 22 14:03:56 zebrafish kernel: block drbd0: State change failed: Device is
> held open by someone
> Jul 22 14:03:56 zebrafish kernel: block drbd0: state = { cs:SyncSource
> ro:Primary/Secondary ds:UpToDate/Inconsistent r----- }
> Jul 22 14:03:56 zebrafish kernel: block drbd0: wanted = { cs:SyncSource
> ro:Secondary/Secondary ds:UpToDate/Inconsistent r----- }
> Jul 22 14:03:56 zebrafish drbd(drbd_storage)[7093]: ERROR: raid: Called
> drbdadm -c /etc/drbd.conf secondary raid
> Jul 22 14:03:56 zebrafish drbd(drbd_storage)[7093]: ERROR: raid: Exit code 11
> Jul 22 14:03:56 zebrafish drbd(drbd_storage)[7093]: ERROR: raid: Command
> output:
...
this last bit: "kernel: block drbd 0"x3 and "drbd(drbd_storage)"x3 goes
on until the power-cycle.
--
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Users mailing list: Users@clusterlabs.org http://lists.clusterlabs.org/mailman/listinfo/users Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
