Public bug reported: We had a network event and at least one controller lost network access for a while. After the network was back I created some VMs. One is stuck in BUILD state. I tried to set it to "ERROR" so that it could be deleted:
$ os server set --state error 09a04278-70f9-4995-b794-61c362f1fe41 Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'oslo_db.exception.DBReferenceError'> (HTTP 500) (Request-ID: req-8cc82bb9-ddc3-4a85-a87d-ca2183934168) We're running kolla-ansible Wallaby with libvirt. Our network is Neutron with openvswitch. Here are the nova-api logs: 2024-09-19 19:41:43.262 50 INFO nova.api.openstack.requestlog [req-2e2699c8-eb85-47ca-9ca3-0858b9b99741 a187a200734f211259c2ca28af7ff440b2616954763d3188d0ca1e5424dce646 076933d285e0424ba846c51e14db006d - b6882f5830c4474d9203b5c5b0ac4ae6 b6882f5830c4474d9203b5c5b0ac4ae6] 10.10.10.12 "GET /v2.1/servers/detail?sort_dir=asc&sort_key=updated_at" status: 200 len: 15 microversion: 2.1 time: 0.059151 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi [req-8cc82bb9-ddc3-4a85-a87d-ca2183934168 9ce5d6afb92d440d8819f61cf47c0dc1 037ee0cfa02645908ce8265b99e08645 - default default] Unexpected exception in API method: oslo_db.exception.DBReferenceError: (pymysql.err.IntegrityError) (1452, 'Cannot add or update a child row: a foreign key constraint fails (`nova_cell0`.`instance_actions`, CONSTRAINT `fk_instance_actions_instance_uuid` FOREIGN KEY (`instance_uuid`) REFERENCES `instances` (`uuid`))') [SQL: INSERT INTO instance_actions (created_at, updated_at, deleted_at, deleted, `action`, instance_uuid, request_id, user_id, project_id, start_time, finish_time, message) VALUES (%(created_at)s, %(updated_at)s, %(deleted_at)s, %(deleted)s, %(action)s, %(instance_uuid)s, %(request_id)s, %(user_id)s, %(project_id)s, %(start_time)s, %(finish_time)s, %(message)s)] [parameters: {'created_at': datetime.datetime(2024, 9, 19, 19, 41, 43, 729593), 'updated_at': datetime.datetime(2024, 9, 19, 19, 41, 43, 710866), 'deleted_at': None, 'deleted': 0, 'action': 'resetState', 'instance_uuid': '09a04278-70f9-4995-b794-61c362f1fe41', 'request_id': 'req-8cc82bb9-ddc3-4a85-a87d-ca2183934168', 'user_id': '9ce5d6afb92d440d8819f61cf47c0dc1', 'project_id': '037ee0cfa02645908ce8265b99e08645', 'start_time': datetime.datetime(2024, 9, 19, 19, 41, 43, 710866), 'finish_time': None, 'message': None}] (Background on this error at: http://sqlalche.me/e/13/gkpj) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi Traceback (most recent call last): 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi self.dialect.do_execute( 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/cursors.py", line 163, in execute 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi result = self._query(query) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/cursors.py", line 321, in _query 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi conn.query(q) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 505, in query 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 724, in _read_query_result 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi result.read() 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 1069, in read 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 676, in _read_packet 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi packet.raise_for_error() 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/protocol.py", line 223, in raise_for_error 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/err.py", line 107, in raise_mysql_exception 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi pymysql.err.IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (`nova_cell0`.`instance_actions`, CONSTRAINT `fk_instance_actions_instance_uuid` FOREIGN KEY (`instance_uuid`) REFERENCES `instances` (`uuid`))') 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception: 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi Traceback (most recent call last): 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/nova/api/openstack/wsgi.py", line 658, in wrapped 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi return f(*args, **kwargs) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/nova/api/validation/__init__.py", line 110, in wrapper 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi return func(*args, **kwargs) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/nova/api/openstack/compute/admin_actions.py", line 71, in _reset_state 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi instance_action = objects.InstanceAction.action_start( 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/oslo_versionedobjects/base.py", line 184, in wrapper 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/nova/objects/instance_action.py", line 86, in action_start 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi db_action = db.action_start(context, values) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/nova/db/api.py", line 1184, in action_start 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi return IMPL.action_start(context, values) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/nova/db/sqlalchemy/api.py", line 221, in wrapped 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi return f(context, *args, **kwargs) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/nova/db/sqlalchemy/api.py", line 3769, in action_start 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi action_ref.save(context.session) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/oslo_db/sqlalchemy/models.py", line 48, in save 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi session.flush() 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/session.py", line 2540, in flush 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi self._flush(objects) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi transaction.rollback(_capture_exception=True) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__ 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi compat.raise_( 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/util/compat.py", line 182, in raise_ 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi raise exception 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi flush_context.execute() 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi rec.execute(self) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi persistence.save_obj( 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi _emit_insert_statements( 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi result = cached_connections[connection].execute( 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1011, in execute 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi return meth(self, multiparams, params) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi return connection._execute_clauseelement(self, multiparams, params) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi ret = self._execute_context( 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi self._handle_dbapi_exception( 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1508, in _handle_dbapi_exception 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi util.raise_(newraise, with_traceback=exc_info[2], from_=e) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/util/compat.py", line 182, in raise_ 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi raise exception 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi self.dialect.do_execute( 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/cursors.py", line 163, in execute 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi result = self._query(query) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/cursors.py", line 321, in _query 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi conn.query(q) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 505, in query 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 724, in _read_query_result 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi result.read() 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 1069, in read 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 676, in _read_packet 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi packet.raise_for_error() 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/protocol.py", line 223, in raise_for_error 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/err.py", line 107, in raise_mysql_exception 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi oslo_db.exception.DBReferenceError: (pymysql.err.IntegrityError) (1452, 'Cannot add or update a child row: a foreign key constraint fails (`nova_cell0`.`instance_actions`, CONSTRAINT `fk_instance_actions_instance_uuid` FOREIGN KEY (`instance_uuid`) REFERENCES `instances` (`uuid`))') 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi [SQL: INSERT INTO instance_actions (created_at, updated_at, deleted_at, deleted, `action`, instance_uuid, request_id, user_id, project_id, start_time, finish_time, message) VALUES (%(created_at)s, %(updated_at)s, %(deleted_at)s, %(deleted)s, %(action)s, %(instance_uuid)s, %(request_id)s, %(user_id)s, %(project_id)s, %(start_time)s, %(finish_time)s, %(message)s)] 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi [parameters: {'created_at': datetime.datetime(2024, 9, 19, 19, 41, 43, 729593), 'updated_at': datetime.datetime(2024, 9, 19, 19, 41, 43, 710866), 'deleted_at': None, 'deleted': 0, 'action': 'resetState', 'instance_uuid': '09a04278-70f9-4995-b794-61c362f1fe41', 'request_id': 'req-8cc82bb9-ddc3-4a85-a87d-ca2183934168', 'user_id': '9ce5d6afb92d440d8819f61cf47c0dc1', 'project_id': '037ee0cfa02645908ce8265b99e08645', 'start_time': datetime.datetime(2024, 9, 19, 19, 41, 43, 710866), 'finish_time': None, 'message': None}] 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi (Background on this error at: http://sqlalche.me/e/13/gkpj) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi 2024-09-19 19:41:43.736 26 INFO nova.api.openstack.wsgi [req-8cc82bb9-ddc3-4a85-a87d-ca2183934168 9ce5d6afb92d440d8819f61cf47c0dc1 037ee0cfa02645908ce8265b99e08645 - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'oslo_db.exception.DBReferenceError'> 2024-09-19 19:41:43.737 26 INFO nova.api.openstack.requestlog [req-8cc82bb9-ddc3-4a85-a87d-ca2183934168 9ce5d6afb92d440d8819f61cf47c0dc1 037ee0cfa02645908ce8265b99e08645 - default default] 10.8.8.41 "POST /v2.1/servers/09a04278-70f9-4995-b794-61c362f1fe41/action" status: 500 len: 208 microversion: 2.1 time: 0.028148 ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/2081216 Title: openstack server set fails Status in OpenStack Compute (nova): New Bug description: We had a network event and at least one controller lost network access for a while. After the network was back I created some VMs. One is stuck in BUILD state. I tried to set it to "ERROR" so that it could be deleted: $ os server set --state error 09a04278-70f9-4995-b794-61c362f1fe41 Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'oslo_db.exception.DBReferenceError'> (HTTP 500) (Request-ID: req-8cc82bb9-ddc3-4a85-a87d-ca2183934168) We're running kolla-ansible Wallaby with libvirt. Our network is Neutron with openvswitch. Here are the nova-api logs: 2024-09-19 19:41:43.262 50 INFO nova.api.openstack.requestlog [req-2e2699c8-eb85-47ca-9ca3-0858b9b99741 a187a200734f211259c2ca28af7ff440b2616954763d3188d0ca1e5424dce646 076933d285e0424ba846c51e14db006d - b6882f5830c4474d9203b5c5b0ac4ae6 b6882f5830c4474d9203b5c5b0ac4ae6] 10.10.10.12 "GET /v2.1/servers/detail?sort_dir=asc&sort_key=updated_at" status: 200 len: 15 microversion: 2.1 time: 0.059151 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi [req-8cc82bb9-ddc3-4a85-a87d-ca2183934168 9ce5d6afb92d440d8819f61cf47c0dc1 037ee0cfa02645908ce8265b99e08645 - default default] Unexpected exception in API method: oslo_db.exception.DBReferenceError: (pymysql.err.IntegrityError) (1452, 'Cannot add or update a child row: a foreign key constraint fails (`nova_cell0`.`instance_actions`, CONSTRAINT `fk_instance_actions_instance_uuid` FOREIGN KEY (`instance_uuid`) REFERENCES `instances` (`uuid`))') [SQL: INSERT INTO instance_actions (created_at, updated_at, deleted_at, deleted, `action`, instance_uuid, request_id, user_id, project_id, start_time, finish_time, message) VALUES (%(created_at)s, %(updated_at)s, %(deleted_at)s, %(deleted)s, %(action)s, %(instance_uuid)s, %(request_id)s, %(user_id)s, %(project_id)s, %(start_time)s, %(finish_time)s, %(message)s)] [parameters: {'created_at': datetime.datetime(2024, 9, 19, 19, 41, 43, 729593), 'updated_at': datetime.datetime(2024, 9, 19, 19, 41, 43, 710866), 'deleted_at': None, 'deleted': 0, 'action': 'resetState', 'instance_uuid': '09a04278-70f9-4995-b794-61c362f1fe41', 'request_id': 'req-8cc82bb9-ddc3-4a85-a87d-ca2183934168', 'user_id': '9ce5d6afb92d440d8819f61cf47c0dc1', 'project_id': '037ee0cfa02645908ce8265b99e08645', 'start_time': datetime.datetime(2024, 9, 19, 19, 41, 43, 710866), 'finish_time': None, 'message': None}] (Background on this error at: http://sqlalche.me/e/13/gkpj) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi Traceback (most recent call last): 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi self.dialect.do_execute( 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/cursors.py", line 163, in execute 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi result = self._query(query) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/cursors.py", line 321, in _query 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi conn.query(q) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 505, in query 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 724, in _read_query_result 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi result.read() 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 1069, in read 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 676, in _read_packet 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi packet.raise_for_error() 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/protocol.py", line 223, in raise_for_error 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/err.py", line 107, in raise_mysql_exception 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi pymysql.err.IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (`nova_cell0`.`instance_actions`, CONSTRAINT `fk_instance_actions_instance_uuid` FOREIGN KEY (`instance_uuid`) REFERENCES `instances` (`uuid`))') 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception: 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi Traceback (most recent call last): 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/nova/api/openstack/wsgi.py", line 658, in wrapped 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi return f(*args, **kwargs) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/nova/api/validation/__init__.py", line 110, in wrapper 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi return func(*args, **kwargs) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/nova/api/openstack/compute/admin_actions.py", line 71, in _reset_state 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi instance_action = objects.InstanceAction.action_start( 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/oslo_versionedobjects/base.py", line 184, in wrapper 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/nova/objects/instance_action.py", line 86, in action_start 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi db_action = db.action_start(context, values) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/nova/db/api.py", line 1184, in action_start 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi return IMPL.action_start(context, values) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/nova/db/sqlalchemy/api.py", line 221, in wrapped 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi return f(context, *args, **kwargs) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/nova/db/sqlalchemy/api.py", line 3769, in action_start 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi action_ref.save(context.session) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/oslo_db/sqlalchemy/models.py", line 48, in save 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi session.flush() 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/session.py", line 2540, in flush 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi self._flush(objects) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi transaction.rollback(_capture_exception=True) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__ 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi compat.raise_( 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/util/compat.py", line 182, in raise_ 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi raise exception 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi flush_context.execute() 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi rec.execute(self) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi persistence.save_obj( 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi _emit_insert_statements( 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi result = cached_connections[connection].execute( 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1011, in execute 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi return meth(self, multiparams, params) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi return connection._execute_clauseelement(self, multiparams, params) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi ret = self._execute_context( 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi self._handle_dbapi_exception( 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1508, in _handle_dbapi_exception 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi util.raise_(newraise, with_traceback=exc_info[2], from_=e) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/util/compat.py", line 182, in raise_ 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi raise exception 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi self.dialect.do_execute( 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib64/python3.9/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/cursors.py", line 163, in execute 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi result = self._query(query) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/cursors.py", line 321, in _query 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi conn.query(q) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 505, in query 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 724, in _read_query_result 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi result.read() 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 1069, in read 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/connections.py", line 676, in _read_packet 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi packet.raise_for_error() 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/protocol.py", line 223, in raise_for_error 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.9/site-packages/pymysql/err.py", line 107, in raise_mysql_exception 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi oslo_db.exception.DBReferenceError: (pymysql.err.IntegrityError) (1452, 'Cannot add or update a child row: a foreign key constraint fails (`nova_cell0`.`instance_actions`, CONSTRAINT `fk_instance_actions_instance_uuid` FOREIGN KEY (`instance_uuid`) REFERENCES `instances` (`uuid`))') 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi [SQL: INSERT INTO instance_actions (created_at, updated_at, deleted_at, deleted, `action`, instance_uuid, request_id, user_id, project_id, start_time, finish_time, message) VALUES (%(created_at)s, %(updated_at)s, %(deleted_at)s, %(deleted)s, %(action)s, %(instance_uuid)s, %(request_id)s, %(user_id)s, %(project_id)s, %(start_time)s, %(finish_time)s, %(message)s)] 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi [parameters: {'created_at': datetime.datetime(2024, 9, 19, 19, 41, 43, 729593), 'updated_at': datetime.datetime(2024, 9, 19, 19, 41, 43, 710866), 'deleted_at': None, 'deleted': 0, 'action': 'resetState', 'instance_uuid': '09a04278-70f9-4995-b794-61c362f1fe41', 'request_id': 'req-8cc82bb9-ddc3-4a85-a87d-ca2183934168', 'user_id': '9ce5d6afb92d440d8819f61cf47c0dc1', 'project_id': '037ee0cfa02645908ce8265b99e08645', 'start_time': datetime.datetime(2024, 9, 19, 19, 41, 43, 710866), 'finish_time': None, 'message': None}] 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi (Background on this error at: http://sqlalche.me/e/13/gkpj) 2024-09-19 19:41:43.732 26 ERROR nova.api.openstack.wsgi 2024-09-19 19:41:43.736 26 INFO nova.api.openstack.wsgi [req-8cc82bb9-ddc3-4a85-a87d-ca2183934168 9ce5d6afb92d440d8819f61cf47c0dc1 037ee0cfa02645908ce8265b99e08645 - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'oslo_db.exception.DBReferenceError'> 2024-09-19 19:41:43.737 26 INFO nova.api.openstack.requestlog [req-8cc82bb9-ddc3-4a85-a87d-ca2183934168 9ce5d6afb92d440d8819f61cf47c0dc1 037ee0cfa02645908ce8265b99e08645 - default default] 10.8.8.41 "POST /v2.1/servers/09a04278-70f9-4995-b794-61c362f1fe41/action" status: 500 len: 208 microversion: 2.1 time: 0.028148 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/2081216/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
