Public bug reported: Description ===========
I8af2ad741ca08c3d88efb9aa817c4d1470491a23 has started to correctly fence the subnode during evacuation testing. However it missed that we deploy c-vol and g-api on these nodes. As a result during BFV evacuation testing we will fail if the volume has been created on the subnode c-vol. https://zuul.opendev.org/t/openstack/build/c78d3ab4e6a748b4a53c6ff6dc273106/log/logs/screen-n-cpu.txt#7060 Mar 19 19:43:26.844295 ubuntu-bionic-rax-ord-0015339373 nova- compute[9838]: ERROR nova.compute.manager [None req- 512a96c8-8b32-49c7-8d29-7ff300ed4482 demo admin] [instance: 702ff125-d947-4a28-853b-82dcd58b990e] Setting instance vm_state to ERROR: ClientException: The server has either erred or is incapable of performing the requested operation. (HTTP 500) https://zuul.opendev.org/t/openstack/build/c78d3ab4e6a748b4a53c6ff6dc273106/log/logs/screen-c-api.txt#1936 Mar 19 19:43:26.262818 ubuntu-bionic-rax-ord-0015339373 [email protected][27200]: ERROR cinder.api.middleware.fault [req-512a96c8-8b32-49c7-8d29-7ff300ed4482 req-826f7c01-3c02-4d9e-9046-8a15d7fa9b61 demo admin] Caught error: <class 'oslo_messaging.exceptions.MessagingTimeout'> Timed out waiting for a reply to message ID 23fabce9b79441198fbe4fe71c0ac7ab: MessagingTimeout: Timed out waiting for a reply to message ID 23fabce9b79441198fbe4fe71c0ac7ab Mar 19 19:43:26.262818 ubuntu-bionic-rax-ord-0015339373 [email protected][27200]: ERROR Ultimately we shouldn't run these services on the computes but for now we should limit the services we stop on the subnode to n-cpu and q-agt. Steps to reproduce ================== Run nova-live-migration, if volumes are created on the subnode evacuation testing will fail. Expected result =============== nova-live-migration passes. Actual result ============= nova-live-migration fails. Environment =========== 1. Exact version of OpenStack you are running. See the following list for all releases: http://docs.openstack.org/releases/ Master or stabe/train with I8af2ad741ca08c3d88efb9aa817c4d1470491a23 applied. 2. Which hypervisor did you use? (For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...) What's the version of that? Libvirt + KVM 2. Which storage type did you use? (For example: Ceph, LVM, GPFS, ...) What's the version of that? N/A 3. Which networking type did you use? (For example: nova-network, Neutron with OpenVSwitch, ...) N/A ** Affects: nova Importance: High Assignee: Lee Yarwood (lyarwood) Status: In Progress ** Affects: nova/stein Importance: Undecided Status: New ** Affects: nova/train Importance: Undecided Status: New ** Affects: nova/ussuri Importance: High Assignee: Lee Yarwood (lyarwood) Status: In Progress ** Tags: evacuate live-migration volumes -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1868234 Title: nova-live-migration evacuation fails if volumes created on subnode c-vol backend Status in OpenStack Compute (nova): In Progress Status in OpenStack Compute (nova) stein series: New Status in OpenStack Compute (nova) train series: New Status in OpenStack Compute (nova) ussuri series: In Progress Bug description: Description =========== I8af2ad741ca08c3d88efb9aa817c4d1470491a23 has started to correctly fence the subnode during evacuation testing. However it missed that we deploy c-vol and g-api on these nodes. As a result during BFV evacuation testing we will fail if the volume has been created on the subnode c-vol. https://zuul.opendev.org/t/openstack/build/c78d3ab4e6a748b4a53c6ff6dc273106/log/logs/screen-n-cpu.txt#7060 Mar 19 19:43:26.844295 ubuntu-bionic-rax-ord-0015339373 nova- compute[9838]: ERROR nova.compute.manager [None req- 512a96c8-8b32-49c7-8d29-7ff300ed4482 demo admin] [instance: 702ff125-d947-4a28-853b-82dcd58b990e] Setting instance vm_state to ERROR: ClientException: The server has either erred or is incapable of performing the requested operation. (HTTP 500) https://zuul.opendev.org/t/openstack/build/c78d3ab4e6a748b4a53c6ff6dc273106/log/logs/screen-c-api.txt#1936 Mar 19 19:43:26.262818 ubuntu-bionic-rax-ord-0015339373 [email protected][27200]: ERROR cinder.api.middleware.fault [req-512a96c8-8b32-49c7-8d29-7ff300ed4482 req-826f7c01-3c02-4d9e-9046-8a15d7fa9b61 demo admin] Caught error: <class 'oslo_messaging.exceptions.MessagingTimeout'> Timed out waiting for a reply to message ID 23fabce9b79441198fbe4fe71c0ac7ab: MessagingTimeout: Timed out waiting for a reply to message ID 23fabce9b79441198fbe4fe71c0ac7ab Mar 19 19:43:26.262818 ubuntu-bionic-rax-ord-0015339373 [email protected][27200]: ERROR Ultimately we shouldn't run these services on the computes but for now we should limit the services we stop on the subnode to n-cpu and q-agt. Steps to reproduce ================== Run nova-live-migration, if volumes are created on the subnode evacuation testing will fail. Expected result =============== nova-live-migration passes. Actual result ============= nova-live-migration fails. Environment =========== 1. Exact version of OpenStack you are running. See the following list for all releases: http://docs.openstack.org/releases/ Master or stabe/train with I8af2ad741ca08c3d88efb9aa817c4d1470491a23 applied. 2. Which hypervisor did you use? (For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...) What's the version of that? Libvirt + KVM 2. Which storage type did you use? (For example: Ceph, LVM, GPFS, ...) What's the version of that? N/A 3. Which networking type did you use? (For example: nova-network, Neutron with OpenVSwitch, ...) N/A To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1868234/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
