On 10/10/2016 10:17 AM, Pavel Levshin wrote: > Hello. > > We are trying to migrate our services to relatively fresh version of > cluster software. It is RHEL 7 with pacemaker 1.1.13-10. I’ve faced a > problem when live migration of virtual machines is allowed. In short, > I need to manage libvirtd, and I cannot set proper ordering > constraints to stop libvirtd only after all virtual machines are > stopped. This leads to failing migration and STONITH when I try to > shutdown one of the nodes. > > If live migration Is disabled, then all works well. Why are the order-constraints between libvirt & vms optional? > > Cluster config follows: > > == > > Cluster Name: smartbvcluster > > Corosync Nodes: > > bvnode1 bvnode2 > > Pacemaker Nodes: > > bvnode1 bvnode2 > > Resources: > > Clone: dlm-clone > > Meta Attrs: interleave=true ordered=true > > Resource: dlm (class=ocf provider=pacemaker type=controld) > > Operations: start interval=0s timeout=90 (dlm-start-interval-0s) > > stop interval=0s timeout=100 (dlm-stop-interval-0s) > > monitor interval=30s (dlm-monitor-interval-30s) > > Clone: clvmd-clone > > Meta Attrs: interleave=true ordered=true > > Resource: clvmd (class=ocf provider=heartbeat type=clvm) > > Operations: start interval=0s timeout=90 (clvmd-start-interval-0s) > > stop interval=0s timeout=90 (clvmd-stop-interval-0s) > > monitor interval=30s (clvmd-monitor-interval-30s) > > Clone: cluster-config-clone > > Meta Attrs: interleave=true > > Resource: cluster-config (class=ocf provider=heartbeat type=Filesystem) > > Attributes: device=/dev/vg_bv_shared/cluster-config > directory=/opt/cluster-config fstype=gfs2 options=noatime > > Operations: start interval=0s timeout=60 > (cluster-config-start-interval-0s) > > stop interval=0s timeout=60 > (cluster-config-stop-interval-0s) > > monitor interval=10s on-fail=fence OCF_CHECK_LEVEL=20 > (cluster-config-monitor-interval-10s) > > Resource: vm_smartbv1 (class=ocf provider=heartbeat type=VirtualDomain) > > Attributes: config=/opt/cluster-config/libvirt/qemu/smartbv1.xml > hypervisor=qemu:///system migration_transport=tcp > > Meta Attrs: allow-migrate=true > > Operations: start interval=0s timeout=90 > (vm_smartbv1-start-interval-0s) > > stop interval=0s timeout=90 (vm_smartbv1-stop-interval-0s) > > monitor interval=10 timeout=30 > (vm_smartbv1-monitor-interval-10) > > Resource: vm_smartbv2 (class=ocf provider=heartbeat type=VirtualDomain) > > Attributes: config=/opt/cluster-config/libvirt/qemu/smartbv2.xml > hypervisor=qemu:///system migration_transport=tcp > > Meta Attrs: target-role=started allow-migrate=true > > Operations: start interval=0s timeout=90 > (vm_smartbv2-start-interval-0s) > > stop interval=0s timeout=90 (vm_smartbv2-stop-interval-0s) > > monitor interval=10 timeout=30 > (vm_smartbv2-monitor-interval-10) > > Clone: libvirtd-clone > > Meta Attrs: interleave=true > > Resource: libvirtd (class=systemd type=libvirtd) > > Operations: monitor interval=60s (libvirtd-monitor-interval-60s) > > Stonith Devices: > > Resource: ilo.bvnode2 (class=stonith type=fence_ilo4) > > Attributes: ipaddr=ilo.bvnode2 login=hacluster passwd=s > pcmk_host_list=bvnode2 privlvl=operator > > Operations: monitor interval=60s (ilo.bvnode2-monitor-interval-60s) > > Resource: ilo.bvnode1 (class=stonith type=fence_ilo4) > > Attributes: ipaddr=ilo.bvnode1 login=hacluster passwd=s > pcmk_host_list=bvnode1 privlvl=operator > > Operations: monitor interval=60s (ilo.bvnode1-monitor-interval-60s) > > Fencing Levels: > > Node: bvnode1 > > Level 10 - ilo.bvnode1 > > Node: bvnode2 > > Level 10 - ilo.bvnode2 > > Location Constraints: > > Resource: ilo.bvnode1 > > Disabled on: bvnode1 (score:-INFINITY) > (id:location-ilo.bvnode1-bvnode1--INFINITY) > > Resource: ilo.bvnode2 > > Disabled on: bvnode2 (score:-INFINITY) > (id:location-ilo.bvnode2-bvnode2--INFINITY) > > Ordering Constraints: > > start dlm-clone then start clvmd-clone (kind:Mandatory) > (id:order-dlm-clone-clvmd-clone-mandatory) > > start clvmd-clone then start cluster-config-clone (kind:Mandatory) > (id:order-clvmd-clone-cluster-config-clone-mandatory) > > start cluster-config-clone then start libvirtd-clone > (kind:Mandatory) (id:order-cluster-config-clone-libvirtd-clone-mandatory) > > start libvirtd-clone then start vm_smartbv2 (kind:Optional) > (id:order-libvirtd-clone-vm_smartbv2-Optional) > > start libvirtd-clone then start vm_smartbv1 (kind:Optional) > (id:order-libvirtd-clone-vm_smartbv1-Optional) > > Colocation Constraints: > > clvmd-clone with dlm-clone (score:INFINITY) > (id:colocation-clvmd-clone-dlm-clone-INFINITY) > > cluster-config-clone with clvmd-clone (score:INFINITY) > (id:colocation-cluster-config-clone-clvmd-clone-INFINITY) > > libvirtd-clone with cluster-config-clone (score:INFINITY) > (id:colocation-libvirtd-clone-cluster-config-clone-INFINITY) > > vm_smartbv1 with libvirtd-clone (score:INFINITY) > (id:colocation-vm_smartbv1-libvirtd-clone-INFINITY) > > vm_smartbv2 with libvirtd-clone (score:INFINITY) > (id:colocation-vm_smartbv2-libvirtd-clone-INFINITY) > > Resources Defaults: > > No defaults set > > Operations Defaults: > > No defaults set > > Cluster Properties: > > cluster-infrastructure: corosync > > cluster-name: smartbvcluster > > dc-version: 1.1.13-10.el7_2.4-44eb2dd > > have-watchdog: false > > last-lrm-refresh: 1475752213 > > maintenance-mode: false > > no-quorum-policy: freeze > > start-failure-is-fatal: false > > stonith-enabled: true > > == > > Crm_simulate output is attached. What exactly means “dashed-arrows > indicate dependencies that are not present in the transition graph”? > What shall I do to make them present? > > > -- > > Pavel Levshin > > > > > _______________________________________________ > Users mailing list: [email protected] > http://clusterlabs.org/mailman/listinfo/users > > Project Home: http://www.clusterlabs.org > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > Bugs: http://bugs.clusterlabs.org
_______________________________________________ Users mailing list: [email protected] http://clusterlabs.org/mailman/listinfo/users Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
