Reviewed: https://review.opendev.org/c/openstack/nova/+/814036 Committed: https://opendev.org/openstack/nova/commit/61fc81a6761d34afdfc4a6d1c4c953802fd8a179 Submitter: "Zuul (22348)" Branch: master
commit 61fc81a6761d34afdfc4a6d1c4c953802fd8a179 Author: Balazs Gibizer <[email protected]> Date: Thu Oct 14 18:09:18 2021 +0200 Prevent leaked eventlets to send notifications In out functional tests we run nova services as eventlets. Also those services can spawn there own eventlets for RPC or other parallel processing. The test case executor only sees and tracks the main eventlet where the code of the test case is running. When that is finishes the test executor considers the test case to be finished regardless of the other spawned eventlets. This could lead to leaked eventlets that are running in parallel with later test cases. One way that it can cause trouble is via the global variables in nova.rpc module. Those globals are re-initialized for each test case so they are not directly leaking information between test cases. However if a late eventlet calls nova.rpc.get_versioned_notifier() it will get a totally usable FakeVersionedNotifier object regardless of which test case this notifier is belongs to or which test case the eventlet belongs to. This way the late eventlet can send a notification to the currently running test case and therefore can make it fail. The current case we saw is the following: 1) The test case nova.tests.functional.test_servers.ServersTestV219.test_description_errors creates a server but don't wait for it to reach terminal state (ACTIVE / ERROR). This test case finishes quickly but leaks running eventlets in the background waiting for some RPC call to return. 2) As the test case finished the cleanup code deletes the test case specific setup, including the DB. 3) The test executor moves forward and starts running another test case 4) 60 seconds later the leaked eventlet times out waiting for the RPC call to return and tries doing things, but fails as the DB is already gone. Then it tries to report this as an error notification. It calls nova.rpc.get_versioned_notifier() and gets a fresh notifier that is connected to the currently running test case. Then emits the error notification there. 5) The currently running test case also waits for an error notification to be triggered by the currently running test code. But it gets the notification form the late eventlet first. As the content of the notification does not match with the expectations the currently running test case fails. The late eventlet prints a lot of error about the DB being gone making the troubleshooting pretty hard. This patch proposes a way to fix this by marking each eventlet at spawn time with the id of the test case that was directly or indirectly started it. Then when the NotificationFixture gets a notification it compares the test case id stored in the calling eventlet with the id of the test case initialized the NotificationFixture. If the two ids do not match then the fixture ignores the notification and raises an exception to the caller eventlet to make it terminate. Change-Id: I012dcf63306bae624dc4f66aae6c6d96a20d4327 Closes-Bug: #1946339 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1946339 Title: test_unshelve_offloaded_server_with_qos_port_pci_update_fails Status in OpenStack Compute (nova): Fix Released Bug description: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_4d3/812392/2/check/nova- tox-functional-py38/4d32a96/testr_results.html 2021-10-07 00:06:06,980 ERROR [nova.compute.manager] Instance failed to spawn Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/manager.py", line 6651, in _unshelve_instance update( File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/utils.py", line 1531, in update_pci_request_spec_with_allocated_interface_name raise ex( nova.exception.UnexpectedResourceProviderNameForPCIRequest: Resource provider 125ffd8a-3163-425a-b397-f915c602b3eb used to allocate resources for the pci request 7059503b-a648-40fd-a561-5ca769304bee does not have a properly formatted name. Expected name format is <hostname>:<agentname>:<interfacename>, but got invalid-device-rp-name 2021-10-07 00:06:06,994 INFO [placement.requestlog] 127.0.0.1 "DELETE /placement/allocations/718d8d5b-8d34-4c11-9080-1952cc824086" status: 204 len: 0 microversion: 1.0 2021-10-07 00:06:06,995 INFO [nova.scheduler.client.report] Deleted allocations for instance 718d8d5b-8d34-4c11-9080-1952cc824086 2021-10-07 00:06:07,029 INFO [nova.compute.manager] Successfully reverted task state from spawning on failure for instance. 2021-10-07 00:06:07,052 ERROR [oslo_messaging.rpc.server] Exception during message handling Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming res = self.dispatcher.dispatch(message) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch return self._do_dispatch(endpoint, method, ctxt, args) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch result = func(ctxt, **new_args) File "/home/zuul/src/opendev.org/openstack/nova/nova/exception_wrapper.py", line 71, in wrapped _emit_versioned_exception_notification( File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_utils/excutils.py", line 227, in __exit__ self.force_reraise() File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_utils/excutils.py", line 200, in force_reraise raise self.value File "/home/zuul/src/opendev.org/openstack/nova/nova/exception_wrapper.py", line 63, in wrapped return f(self, context, *args, **kw) File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/manager.py", line 183, in decorated_function LOG.warning("Failed to revert task state for instance. " File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_utils/excutils.py", line 227, in __exit__ self.force_reraise() File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_utils/excutils.py", line 200, in force_reraise raise self.value File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/manager.py", line 154, in decorated_function return function(self, context, *args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/utils.py", line 1433, in decorated_function return function(self, context, *args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/manager.py", line 211, in decorated_function compute_utils.add_instance_fault_from_exc(context, File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_utils/excutils.py", line 227, in __exit__ self.force_reraise() File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_utils/excutils.py", line 200, in force_reraise raise self.value File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/manager.py", line 200, in decorated_function return function(self, context, *args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/manager.py", line 6595, in unshelve_instance do_unshelve_instance() File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_concurrency/lockutils.py", line 360, in inner return f(*args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/manager.py", line 6592, in do_unshelve_instance self._unshelve_instance( File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/manager.py", line 6697, in _unshelve_instance self._nil_out_instance_obj_host_and_node(instance) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_utils/excutils.py", line 227, in __exit__ self.force_reraise() File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_utils/excutils.py", line 200, in force_reraise raise self.value File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/manager.py", line 6651, in _unshelve_instance update( File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/utils.py", line 1531, in update_pci_request_spec_with_allocated_interface_name raise ex( nova.exception.UnexpectedResourceProviderNameForPCIRequest: Resource provider 125ffd8a-3163-425a-b397-f915c602b3eb used to allocate resources for the pci request 7059503b-a648-40fd-a561-5ca769304bee does not have a properly formatted name. Expected name format is <hostname>:<agentname>:<interfacename>, but got invalid-device-rp-name 2021-10-07 00:06:07,183 INFO [oslo_db.sqlalchemy.provision] DROP ALL OBJECTS, BACKEND sqlite:// 2021-10-07 00:06:07,201 INFO [oslo_db.sqlalchemy.provision] DISPOSE ENGINE Engine(sqlite://) }}} stderr: {{{ Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_messaging/_drivers/impl_fake.py", line 207, in _send reply, failure = reply_q.get(timeout=timeout) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/eventlet/queue.py", line 322, in get return waiter.wait() File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/eventlet/queue.py", line 141, in wait return get_hub().switch() File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/eventlet/hubs/hub.py", line 313, in switch return self.greenlet.switch() _queue.Empty During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/manager.py", line 200, in decorated_function return function(self, context, *args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/manager.py", line 2263, in _do_build_and_run_instance self.compute_task_api.build_instances(context, [instance], File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/api.py", line 119, in build_instances self.conductor_compute_rpcapi.build_instances(context, File "/home/zuul/src/opendev.org/openstack/nova/nova/conductor/rpcapi.py", line 390, in build_instances cctxt.cast(context, 'build_instances', **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/fixtures/_fixtures/monkeypatch.py", line 89, in avoid_get return captured_method(*args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_messaging/rpc/client.py", line 189, in call result = self.transport._send( File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_messaging/transport.py", line 123, in _send return self._driver.send(target, ctxt, message, File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_messaging/_drivers/impl_fake.py", line 222, in send return self._send(target, ctxt, message, wait_for_reply, timeout, File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_messaging/_drivers/impl_fake.py", line 213, in _send raise oslo_messaging.MessagingTimeout( oslo_messaging.exceptions.MessagingTimeout: No reply on topic conductor During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1799, in _execute_context self.dialect.do_execute( File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 717, in do_execute cursor.execute(statement, parameters) sqlite3.OperationalError: no such table: instance_faults The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/utils.py", line 1433, in decorated_function return function(self, context, *args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/manager.py", line 211, in decorated_function compute_utils.add_instance_fault_from_exc(context, File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/utils.py", line 153, in add_instance_fault_from_exc fault_obj.create() File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_versionedobjects/base.py", line 226, in wrapper return fn(self, *args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/objects/instance_fault.py", line 76, in create db_fault = db.instance_fault_create(self._context, values) File "/home/zuul/src/opendev.org/openstack/nova/nova/db/main/api.py", line 190, in wrapper return f(context, *args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/db/main/api.py", line 3881, in instance_fault_create fault_ref.save(context.session) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_db/sqlalchemy/models.py", line 48, in save session.flush() File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3339, in flush self._flush(objects) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3479, in _flush transaction.rollback(_capture_exception=True) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ compat.raise_( File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise_ raise exception File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3439, in _flush flush_context.execute() File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute rec.execute(self) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute util.preloaded.orm_persistence.save_obj( File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 242, in save_obj _emit_insert_statements( File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1219, in _emit_insert_statements result = connection._execute_20( File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1611, in _execute_20 return meth(self, args_10style, kwargs_10style, execution_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection return connection._execute_clauseelement( File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1478, in _execute_clauseelement ret = self._execute_context( File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1842, in _execute_context self._handle_dbapi_exception( File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2021, in _handle_dbapi_exception util.raise_(newraise, with_traceback=exc_info[2], from_=e) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise_ raise exception File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1799, in _execute_context self.dialect.do_execute( File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 717, in do_execute cursor.execute(statement, parameters) oslo_db.exception.DBNonExistentTable: (sqlite3.OperationalError) no such table: instance_faults [SQL: INSERT INTO instance_faults (created_at, updated_at, deleted_at, deleted, instance_uuid, code, message, details, host) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)] [parameters: ('2021-10-07 00:06:06.747150', None, None, 0, '5903406a-6fff-4179-99bf-8bab9c33f0cb', 500, 'MessagingTimeout', 'Traceback (most recent call last):\n File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_messaging ... (2344 characters truncated) ... l_fake.py", line 213, in _send\n raise oslo_messaging.MessagingTimeout(\noslo_messaging.exceptions.MessagingTimeout: No reply on topic conductor\n', 'host2')] (Background on this error at: https://sqlalche.me/e/14/e3q8) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1799, in _execute_context self.dialect.do_execute( File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 717, in do_execute cursor.execute(statement, parameters) sqlite3.OperationalError: no such table: instance_actions The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/eventlet/hubs/hub.py", line 476, in fire_timers timer() File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/eventlet/hubs/timer.py", line 59, in __call__ cb(*args, **kw) File "/home/zuul/src/opendev.org/openstack/nova/nova/utils.py", line 680, in context_wrapper func(*args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_concurrency/lockutils.py", line 360, in inner return f(*args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/manager.py", line 2115, in _locked_do_build_and_run_instance result = self._do_build_and_run_instance(*args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/exception_wrapper.py", line 71, in wrapped _emit_versioned_exception_notification( File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_utils/excutils.py", line 227, in __exit__ self.force_reraise() File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_utils/excutils.py", line 200, in force_reraise raise self.value File "/home/zuul/src/opendev.org/openstack/nova/nova/exception_wrapper.py", line 63, in wrapped return f(self, context, *args, **kw) File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/manager.py", line 183, in decorated_function LOG.warning("Failed to revert task state for instance. " File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_utils/excutils.py", line 227, in __exit__ self.force_reraise() File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_utils/excutils.py", line 200, in force_reraise raise self.value File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/manager.py", line 154, in decorated_function return function(self, context, *args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/utils.py", line 1433, in decorated_function return function(self, context, *args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/utils.py", line 1394, in __exit__ objects.InstanceActionEvent.event_finish_with_failure( File "/home/zuul/src/opendev.org/openstack/nova/nova/objects/base.py", line 355, in wrapper return fn.__get__(None, obj)(*args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_versionedobjects/base.py", line 184, in wrapper result = fn(cls, context, *args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/objects/instance_action.py", line 227, in event_finish_with_failure db_event = db.action_event_finish(context, values) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_db/api.py", line 154, in wrapper ectxt.value = e.inner_exc File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_utils/excutils.py", line 227, in __exit__ self.force_reraise() File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_utils/excutils.py", line 200, in force_reraise raise self.value File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/oslo_db/api.py", line 142, in wrapper return f(*args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/db/main/api.py", line 190, in wrapper return f(context, *args, **kwargs) File "/home/zuul/src/opendev.org/openstack/nova/nova/db/main/api.py", line 4078, in action_event_finish action = _action_get_by_request_id(context, values['instance_uuid'], File "/home/zuul/src/opendev.org/openstack/nova/nova/db/main/api.py", line 4014, in _action_get_by_request_id result = model_query(context, models.InstanceAction).\ File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2762, in first return self.limit(1)._iter().first() File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2846, in _iter result = self.session.execute( File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1689, in execute result = conn._execute_20(statement, params or {}, execution_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1611, in _execute_20 return meth(self, args_10style, kwargs_10style, execution_options) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection return connection._execute_clauseelement( File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1478, in _execute_clauseelement ret = self._execute_context( File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1842, in _execute_context self._handle_dbapi_exception( File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2021, in _handle_dbapi_exception util.raise_(newraise, with_traceback=exc_info[2], from_=e) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise_ raise exception File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1799, in _execute_context self.dialect.do_execute( File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 717, in do_execute cursor.execute(statement, parameters) oslo_db.exception.DBNonExistentTable: (sqlite3.OperationalError) no such table: instance_actions [SQL: SELECT instance_actions.created_at AS instance_actions_created_at, instance_actions.updated_at AS instance_actions_updated_at, instance_actions.deleted_at AS instance_actions_deleted_at, instance_actions.deleted AS instance_actions_deleted, instance_actions.id AS instance_actions_id, instance_actions.action AS instance_actions_action, instance_actions.instance_uuid AS instance_actions_instance_uuid, instance_actions.request_id AS instance_actions_request_id, instance_actions.user_id AS instance_actions_user_id, instance_actions.project_id AS instance_actions_project_id, instance_actions.start_time AS instance_actions_start_time, instance_actions.finish_time AS instance_actions_finish_time, instance_actions.message AS instance_actions_message FROM instance_actions WHERE instance_actions.deleted = ? AND instance_actions.instance_uuid = ? AND instance_actions.request_id = ? ORDER BY instance_actions.created_at DESC, instance_actions.id DESC LIMIT ? OFFSET ?] [parameters: (0, '5903406a-6fff-4179-99bf-8bab9c33f0cb', 'req-14ab570f-ce9a-4a2e-a753-db18915fbe14', 1, 0)] (Background on this error at: https://sqlalche.me/e/14/e3q8) }}} Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/nova/nova/tests/functional/test_servers_resource_request.py", line 2491, in test_unshelve_offloaded_server_with_qos_port_pci_update_fails self.assertEqual( File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/testtools/testcase.py", line 393, in assertEqual self.assertThat(observed, matcher, message) File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/testtools/testcase.py", line 480, in assertThat raise mismatch_error testtools.matchers._impl.MismatchError: 'UnexpectedResourceProviderNameForPCIRequest' != 'DBNonExistentTable' To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1946339/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
