Reviewed: https://review.opendev.org/699997 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=40950f9c87681a1db0f3819530dd0b89ac5f6553 Submitter: Zuul Branch: master
commit 40950f9c87681a1db0f3819530dd0b89ac5f6553 Author: Matt Riedemann <[email protected]> Date: Thu Dec 19 09:41:18 2019 -0500 Fix race in test_create_servers_with_vpmem This test creates 2 servers which take up all of the CUSTOM_PMEM_NAMESPACE_SMALL resources on the sole host in the test. Then it tries to create a 3rd server which it expects to fail and it does. Then it deletes the second server to free up one CUSTOM_PMEM_NAMESPACE_SMALL resource on the host and schedules another server which it expects to work. The problem is the test doesn't wait for the second server to be fully deleted so the test intermittently fails when the CUSTOM_PMEM_NAMESPACE_SMALL resource isn't yet freed up when the fourth server goes through scheduling. This adds a simple wait call for the deleted server to actually be gone before creating the fourth server. Change-Id: Id3389038b5fed1e70dd12c7ed9cef2c0950625cf Closes-Bug: #1856902 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1856902 Title: nova.tests.functional.libvirt.test_vpmem.VPMEMTests.test_create_servers_with_vpmem fails intermittently Status in OpenStack Compute (nova): Fix Released Bug description: Seen here: https://748441c9eeeb89eb36db- b17eeb704755458252956ad3afd603fe.ssl.cf5.rackcdn.com/697670/3/gate /nova-tox-functional-py36/54fe49d/testr_results.html.gz ft1.1: nova.tests.functional.libvirt.test_vpmem.VPMEMTests.test_create_servers_with_vpmemtesttools.testresult.real._StringException: pythonlogging:'': {{{ 2019-12-18 17:05:18,534 WARNING [placement.db_api] TransactionFactory already started, not reconfiguring. 2019-12-18 17:05:18,885 INFO [nova.service] Starting conductor node (version 20.1.0) 2019-12-18 17:05:18,917 INFO [nova.service] Starting scheduler node (version 20.1.0) 2019-12-18 17:05:18,981 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/flavors/detail" status: 200 len: 2605 microversion: 2.81 time: 0.020144 2019-12-18 17:05:18,996 INFO [nova.api.openstack.requestlog] 127.0.0.1 "POST /v2.1/flavors" status: 200 len: 464 microversion: 2.81 time: 0.012445 2019-12-18 17:05:19,018 INFO [nova.api.openstack.requestlog] 127.0.0.1 "POST /v2.1/flavors/99512851/os-extra_specs" status: 200 len: 37 microversion: 2.81 time: 0.018592 2019-12-18 17:05:19,035 INFO [nova.virt.driver] Loading compute driver 'libvirt.LibvirtDriver' 2019-12-18 17:05:19,038 WARNING [os_brick.initiator.connectors.remotefs] Connection details not present. RemoteFsClient may not initialize properly. 2019-12-18 17:05:19,040 INFO [nova.service] Starting compute node (version 20.1.0) 2019-12-18 17:05:19,058 WARNING [nova.compute.manager] Compute node host1 not found in the database. If this is the first time this service is starting on this host, then you can ignore this warning. 2019-12-18 17:05:19,063 INFO [nova.compute.manager] Looking for unclaimed instances stuck in BUILDING status for nodes managed by this host 2019-12-18 17:05:19,073 WARNING [nova.compute.manager] No compute node record found for host host1. If this is the first time this service is starting on this host, then you can ignore this warning. 2019-12-18 17:05:19,082 WARNING [nova.compute.resource_tracker] No compute node record for host1:host1 2019-12-18 17:05:19,086 INFO [nova.compute.resource_tracker] Compute node record created for host1:host1 with uuid: 56a25de3-2d91-48a3-9e77-a66f1b5d65b5 2019-12-18 17:05:19,123 INFO [placement.requestlog] 127.0.0.1 "GET /placement/resource_providers?in_tree=56a25de3-2d91-48a3-9e77-a66f1b5d65b5" status: 200 len: 26 microversion: 1.14 2019-12-18 17:05:19,131 INFO [placement.requestlog] 127.0.0.1 "POST /placement/resource_providers" status: 200 len: 828 microversion: 1.20 2019-12-18 17:05:19,132 INFO [nova.scheduler.client.report] [req-54d75b29-ade3-4f2a-8911-d1147ef449fc] Created resource provider record via placement API for resource provider with UUID 56a25de3-2d91-48a3-9e77-a66f1b5d65b5 and name host1. 2019-12-18 17:05:19,133 INFO [nova.virt.libvirt.host] kernel doesn't support AMD SEV 2019-12-18 17:05:19,149 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/resource_classes/CUSTOM_PMEM_NAMESPACE_SMALL" status: 201 len: 0 microversion: 1.7 2019-12-18 17:05:19,159 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/resource_classes/CUSTOM_PMEM_NAMESPACE_4GB" status: 201 len: 0 microversion: 1.7 2019-12-18 17:05:19,173 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/resource_providers/56a25de3-2d91-48a3-9e77-a66f1b5d65b5/inventories" status: 200 len: 665 microversion: 1.26 2019-12-18 17:05:19,181 INFO [placement.requestlog] 127.0.0.1 "GET /placement/traits?name=in:COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_IMAGE_TYPE_AMI,HW_CPU_X86_VMX,HW_CPU_X86_AESNI,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_DEVICE_TAGGING,HW_CPU_HYPERTHREADING,COMPUTE_VOLUME_EXTEND,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_TRUSTED_CERTS,COMPUTE_NODE" status: 200 len: 432 microversion: 1.6 2019-12-18 17:05:19,200 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/resource_providers/56a25de3-2d91-48a3-9e77-a66f1b5d65b5/traits" status: 200 len: 467 microversion: 1.6 2019-12-18 17:05:19,224 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/os-hypervisors?hypervisor_hostname_pattern=host1" status: 200 len: 133 microversion: 2.81 time: 0.019077 2019-12-18 17:05:19,236 INFO [placement.requestlog] 127.0.0.1 "GET /placement/resource_providers/56a25de3-2d91-48a3-9e77-a66f1b5d65b5/inventories" status: 200 len: 665 microversion: 1.0 2019-12-18 17:05:19,253 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/os-hypervisors?hypervisor_hostname_pattern=host1" status: 200 len: 133 microversion: 2.81 time: 0.014420 2019-12-18 17:05:19,255 INFO [nova.tests.functional.libvirt.test_vpmem] booting on host1 2019-12-18 17:05:19,329 INFO [nova.api.openstack.requestlog] 127.0.0.1 "POST /v2.1/servers" status: 202 len: 414 microversion: 2.81 time: 0.071830 2019-12-18 17:05:19,352 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 200 len: 1778 microversion: 2.81 time: 0.019663 2019-12-18 17:05:19,473 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocation_candidates?in_tree=56a25de3-2d91-48a3-9e77-a66f1b5d65b5&limit=1000&required=%21COMPUTE_STATUS_DISABLED&resources=CUSTOM_PMEM_NAMESPACE_SMALL%3A1%2CDISK_GB%3A1%2CMEMORY_MB%3A1024%2CVCPU%3A1" status: 200 len: 1087 microversion: 1.34 2019-12-18 17:05:19,495 INFO [nova.scheduler.host_manager] Host filter forcing available hosts to host1 2019-12-18 17:05:19,503 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 200 len: 19 microversion: 1.28 2019-12-18 17:05:19,530 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/allocations/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 204 len: 0 microversion: 1.34 2019-12-18 17:05:19,724 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 200 len: 265 microversion: 1.28 2019-12-18 17:05:19,731 INFO [nova.compute.claims] Claim successful on node host1 2019-12-18 17:05:19,985 INFO [nova.compute.manager] Took 0.00 seconds to spawn the instance on the hypervisor. 2019-12-18 17:05:20,077 INFO [nova.compute.manager] Took 0.36 seconds to build instance. 2019-12-18 17:05:20,087 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 200 len: 1875 microversion: 2.81 time: 0.227589 2019-12-18 17:05:20,122 INFO [nova.scheduler.host_manager] Received an update from an unknown host 'host1'. Re-created its InstanceList. 2019-12-18 17:05:20,679 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 200 len: 1880 microversion: 2.81 time: 0.087029 2019-12-18 17:05:20,690 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/3a085e4e-52ba-49cc-be7c-6eca2974765a" status: 200 len: 169 microversion: 1.0 2019-12-18 17:05:20,691 INFO [nova.tests.functional.libvirt.test_vpmem] booting on host1 2019-12-18 17:05:20,766 INFO [nova.api.openstack.requestlog] 127.0.0.1 "POST /v2.1/servers" status: 202 len: 414 microversion: 2.81 time: 0.072347 2019-12-18 17:05:20,786 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 1778 microversion: 2.81 time: 0.017267 2019-12-18 17:05:20,902 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocation_candidates?in_tree=56a25de3-2d91-48a3-9e77-a66f1b5d65b5&limit=1000&required=%21COMPUTE_STATUS_DISABLED&resources=CUSTOM_PMEM_NAMESPACE_SMALL%3A1%2CDISK_GB%3A1%2CMEMORY_MB%3A1024%2CVCPU%3A1" status: 200 len: 1090 microversion: 1.34 2019-12-18 17:05:20,922 INFO [nova.scheduler.host_manager] Host filter forcing available hosts to host1 2019-12-18 17:05:20,930 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 19 microversion: 1.28 2019-12-18 17:05:20,955 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/allocations/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 204 len: 0 microversion: 1.34 2019-12-18 17:05:21,131 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 265 microversion: 1.28 2019-12-18 17:05:21,138 INFO [nova.compute.claims] Claim successful on node host1 2019-12-18 17:05:21,391 INFO [nova.compute.manager] Took 0.00 seconds to spawn the instance on the hypervisor. 2019-12-18 17:05:21,450 INFO [nova.compute.manager] Took 0.33 seconds to build instance. 2019-12-18 17:05:21,490 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 1863 microversion: 2.81 time: 0.183892 2019-12-18 17:05:22,075 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 1880 microversion: 2.81 time: 0.080401 2019-12-18 17:05:22,084 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 169 microversion: 1.0 2019-12-18 17:05:22,085 INFO [nova.tests.functional.libvirt.test_vpmem] booting on host1 2019-12-18 17:05:22,162 INFO [nova.api.openstack.requestlog] 127.0.0.1 "POST /v2.1/servers" status: 202 len: 414 microversion: 2.81 time: 0.074114 2019-12-18 17:05:22,183 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/afeb21d7-a89d-40ea-ac6e-28a7b80cec95" status: 200 len: 1778 microversion: 2.81 time: 0.018034 2019-12-18 17:05:22,283 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocation_candidates?in_tree=56a25de3-2d91-48a3-9e77-a66f1b5d65b5&limit=1000&required=%21COMPUTE_STATUS_DISABLED&resources=CUSTOM_PMEM_NAMESPACE_SMALL%3A1%2CDISK_GB%3A1%2CMEMORY_MB%3A1024%2CVCPU%3A1" status: 200 len: 53 microversion: 1.34 2019-12-18 17:05:22,284 INFO [nova.scheduler.manager] Got no allocation candidates from the Placement API. This could be due to insufficient resources or a temporary occurrence as compute nodes start up. 2019-12-18 17:05:22,284 ERROR [nova.conductor.manager] Failed to schedule instances Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/manager.py", line 1412, in schedule_and_build_instances instance_uuids, return_alternates=True) File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/manager.py", line 855, in _schedule_instances return_alternates=return_alternates) File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/client/query.py", line 42, in select_destinations instance_uuids, return_objects, return_alternates) File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations return cctxt.call(ctxt, 'select_destinations', **msg_args) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 181, in call transport_options=self.transport_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/transport.py", line 129, in _send transport_options=transport_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_fake.py", line 222, in send transport_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_fake.py", line 208, in _send raise failure File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 235, in inner return func(*args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/manager.py", line 199, in select_destinations raise exception.NoValidHost(reason="") nova.exception.NoValidHost: No valid host was found. 2019-12-18 17:05:22,325 WARNING [nova.scheduler.utils] Failed to compute_task_build_instances: No valid host was found. 2019-12-18 17:05:22,325 WARNING [nova.scheduler.utils] Setting instance to ERROR state. 2019-12-18 17:05:22,773 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/afeb21d7-a89d-40ea-ac6e-28a7b80cec95" status: 200 len: 3749 microversion: 2.81 time: 0.084113 2019-12-18 17:05:22,899 INFO [nova.api.openstack.requestlog] 127.0.0.1 "DELETE /v2.1/servers/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 204 len: 0 microversion: 2.81 time: 0.122724 2019-12-18 17:05:22,901 INFO [nova.tests.functional.libvirt.test_vpmem] booting on host1 2019-12-18 17:05:22,952 INFO [nova.compute.manager] Terminating instance 2019-12-18 17:05:22,968 INFO [nova.virt.libvirt.driver] Instance destroyed successfully. 2019-12-18 17:05:22,976 INFO [nova.virt.libvirt.driver] Deletion of /home/zuul/src/opendev.org/openstack/nova/instances/68f4cbad-7c69-492a-a173-6869dfa312d7_del complete 2019-12-18 17:05:23,042 INFO [nova.compute.manager] Took 0.08 seconds to destroy the instance on the hypervisor. 2019-12-18 17:05:23,051 INFO [nova.api.openstack.requestlog] 127.0.0.1 "POST /v2.1/servers" status: 202 len: 414 microversion: 2.81 time: 0.147582 2019-12-18 17:05:23,077 INFO [nova.compute.manager] Took 0.02 seconds to deallocate network for instance. 2019-12-18 17:05:23,095 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 1778 microversion: 2.81 time: 0.040906 2019-12-18 17:05:23,179 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocations/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 200 len: 265 microversion: 1.28 2019-12-18 17:05:23,204 INFO [placement.requestlog] 127.0.0.1 "GET /placement/allocation_candidates?in_tree=56a25de3-2d91-48a3-9e77-a66f1b5d65b5&limit=1000&required=%21COMPUTE_STATUS_DISABLED&resources=CUSTOM_PMEM_NAMESPACE_SMALL%3A1%2CDISK_GB%3A1%2CMEMORY_MB%3A1024%2CVCPU%3A1" status: 200 len: 53 microversion: 1.34 2019-12-18 17:05:23,205 INFO [nova.scheduler.manager] Got no allocation candidates from the Placement API. This could be due to insufficient resources or a temporary occurrence as compute nodes start up. 2019-12-18 17:05:23,208 ERROR [nova.conductor.manager] Failed to schedule instances Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/manager.py", line 1412, in schedule_and_build_instances instance_uuids, return_alternates=True) File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/manager.py", line 855, in _schedule_instances return_alternates=return_alternates) File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/client/query.py", line 42, in select_destinations instance_uuids, return_objects, return_alternates) File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations return cctxt.call(ctxt, 'select_destinations', **msg_args) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 181, in call transport_options=self.transport_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/transport.py", line 129, in _send transport_options=transport_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_fake.py", line 222, in send transport_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_fake.py", line 208, in _send raise failure File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 235, in inner return func(*args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/manager.py", line 199, in select_destinations raise exception.NoValidHost(reason="") nova.exception.NoValidHost: No valid host was found. 2019-12-18 17:05:23,240 INFO [placement.requestlog] 127.0.0.1 "PUT /placement/allocations/68f4cbad-7c69-492a-a173-6869dfa312d7" status: 204 len: 0 microversion: 1.28 2019-12-18 17:05:23,241 INFO [nova.scheduler.client.report] Deleted allocation for instance 68f4cbad-7c69-492a-a173-6869dfa312d7 2019-12-18 17:05:23,312 WARNING [nova.scheduler.utils] Failed to compute_task_build_instances: No valid host was found. 2019-12-18 17:05:23,313 WARNING [nova.scheduler.utils] Setting instance to ERROR state. 2019-12-18 17:05:23,696 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.094132 2019-12-18 17:05:24,254 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.053813 2019-12-18 17:05:24,817 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.058160 2019-12-18 17:05:25,376 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.054598 2019-12-18 17:05:25,943 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.062802 2019-12-18 17:05:26,502 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.054831 2019-12-18 17:05:27,067 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.060461 2019-12-18 17:05:27,629 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.057168 2019-12-18 17:05:28,199 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847" status: 200 len: 3749 microversion: 2.81 time: 0.065318 2019-12-18 17:05:28,339 INFO [oslo_db.sqlalchemy.provision] DROP ALL OBJECTS, BACKEND sqlite:// 2019-12-18 17:05:28,372 INFO [oslo_db.sqlalchemy.provision] DISPOSE ENGINE Engine(sqlite://) }}} Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/nova/nova/tests/functional/libvirt/test_vpmem.py", line 191, in test_create_servers_with_vpmem self._wait_for_state_change(server3, 'ACTIVE') File "/home/zuul/src/opendev.org/openstack/nova/nova/tests/functional/integrated_helpers.py", line 98, in _wait_for_state_change server, {'status': expected_status}, max_retries) File "/home/zuul/src/opendev.org/openstack/nova/nova/tests/functional/integrated_helpers.py", line 91, in _wait_for_server_parameter expected_params, server)) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/unittest2/case.py", line 690, in fail raise self.failureException(msg) AssertionError: Wait for state change failed, expected_params={'status': 'ACTIVE'}, server={'id': 'e67add72-1327-466a-8f96-7e5025fb9847', 'name': 'some-server', 'status': 'ERROR', 'tenant_id': '6f70656e737461636b20342065766572', 'user_id': 'admin', 'metadata': {}, 'hostId': '', 'image': {'id': '155d900f-4e14-4e4c-a73d-069cbf4541e6', 'links': [{'rel': 'bookmark', 'href': 'http://2a542091-42cb-4ba0-a2eb-9ed61b4b29da/images/155d900f-4e14-4e4c-a73d-069cbf4541e6'}]}, 'flavor': {'vcpus': 1, 'ram': 1024, 'disk': 1, 'ephemeral': 0, 'swap': 0, 'original_name': 'flavorBGI1I584', 'extra_specs': {'hw:pmem': 'SMALL'}}, 'created': '2019-12-18T17:05:23Z', 'updated': '2019-12-18T17:05:23Z', 'addresses': {}, 'accessIPv4': '', 'accessIPv6': '', 'links': [{'rel': 'self', 'href': 'http://2a542091-42cb-4ba0-a2eb-9ed61b4b29da/v2.1/servers/e67add72-1327-466a-8f96-7e5025fb9847'}, {'rel': 'bookmark', 'href': 'http://2a542091-42cb-4ba0-a2eb-9ed61b4b29da/servers/e67add72-1327-466a-8f96-7e5025fb9847'}], 'OS-DCF:diskConfig': 'MANUAL', 'fault': {'code': 500, 'created': '2019-12-18T17:05:23Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/manager.py", line 1412, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/manager.py", line 855, in _schedule_instances\n return_alternates=return_alternates)\n File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 181, in call\n transport_options=self.transport_options)\n File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/transport.py", line 129, in _send\n transport_options=transport_options)\n File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_fake.py", line 222, in send\n transport_options)\n File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/_drivers/impl_fake.py", line 208, in _send\n raise failure\n File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py36/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 235, in inner\n return func(*args, **kwargs)\n File "/home/zuul/src/opendev.org/openstack/nova/nova/scheduler/manager.py", line 199, in select_destinations\n raise exception.NoValidHost(reason="")\nnova.exception.NoValidHost: No valid host was found. \n'}, 'OS-EXT-AZ:availability_zone': 'nova', 'config_drive': '', 'key_name': None, 'OS-SRV-USG:launched_at': None, 'OS-SRV-USG:terminated_at': None, 'OS-EXT-SRV-ATTR:host': None, 'OS-EXT-SRV-ATTR:instance_name': 'instance-00000002', 'OS-EXT-SRV-ATTR:hypervisor_hostname': None, 'OS-EXT-SRV-ATTR:reservation_id': 'r-e4t4h5f5', 'OS-EXT-SRV-ATTR:launch_index': 0, 'OS-EXT-SRV-ATTR:hostname': 'some-server', 'OS-EXT-SRV-ATTR:kernel_id': '', 'OS-EXT-SRV-ATTR:ramdisk_id': '', 'OS-EXT-SRV-ATTR:root_device_name': None, 'OS-EXT-SRV-ATTR:user_data': None, 'OS-EXT-STS:task_state': None, 'OS-EXT-STS:vm_state': 'error', 'OS-EXT-STS:power_state': 0, 'os-extended-volumes:volumes_attached': [], 'host_status': '', 'locked': False, 'locked_reason': None, 'description': None, 'tags': [], 'trusted_image_certificates': None, 'server_groups': []} http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22'extra_specs'%3A%20%7B'hw%3Apmem'%3A%20'SMALL'%7D%7D%5C%22%20AND%20tags%3A%5C%22console%5C%22&from=7d 18 hits in 7 days, check and gate, all failures. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1856902/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
