Public bug reported: Description =========== nova delete raise "NO server with a name or ID of 8150bb45-7758-4112-8593-e602b33ab4a5 exists", but this instance is queried from "nova list" command.
Steps to reproduce ================== A chronological list of steps which will bring off the issue you noticed: * I did create some instances, instance with uuid 8150bb45-7758-4112-8593-e602b33ab4a5 is scheduling all the time. nova list --all | grep scheduling | 8150bb45-7758-4112-8593-e602b33ab4a5 | test_aa | BUILD | scheduling | NOSTATE | * then I did delete this instance: nova delete 8150bb45-7758-4112-8593-e602b33ab4a5 NO server with a name or ID of 8150bb45-7758-4112-8593-e602b33ab4a5 exists Error(commandError): unable to delete this specified server(s). Expected result =============== Delete operation can success. Actual result ============= Delete operation failed. Instance information in Database is inconsistent; I look at database, instance's cell is None, can't see this instance in build_request table. And this instance's detail information was found in cell0 database. Environment =========== 1. Exact version of OpenStack you are running. See the following list for all releases: http://docs.openstack.org/releases/ Pike Logs & Configs ============== [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming\n res = self.dispatcher.dispatch(message)\n', u' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', u' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch\n result = func(ctxt, **new_args)\n', u' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 232, in inner\n return func(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 153, in select_destinations\n alloc_reqs_by_rp_uuid, provider_summaries)\n', u' File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 89, in select_destinations\n alloc_reqs_by_rp_uuid, provider_summaries)\n', u' File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 163, in _schedule\n provider_summaries)\n', u' File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 344, in _get_all_host_states\n spec_obj)\n', u' File "/usr/lib/python2.7/site-packages/nova/scheduler/host_manager.py", line 659, in get_host_states_by_uuids\n context, cells, compute_uuids=compute_uuids)\n', u' File "/usr/lib/python2.7/site-packages/nova/scheduler/host_manager.py", line 627, in _get_computes_for_cells\n cctxt, compute_uuids))\n', u' File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 184, in wrapper\n result = fn(cls, context, *args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/nova/objects/compute_node.py", line 460, in get_all_by_uuids\n compute_uuids)\n', u' File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 235, in wrapper\n return f(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/nova/objects/compute_node.py", line 454, in _db_compute_node_get_all_by_uuids\n models.ComputeNode.uuid.in_(compute_uuids)).all()\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2703, in all\n return list(self)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2855, in __iter__\n return self._execute_and_instances(context)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2876, in _execute_and_instances\n close_with_result=True)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2885, in _get_bind_args\n **kw\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2867, in _connection_from_session\n conn = self.session.connection(**kw)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 998, in connection\n execution_options=execution_options)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1003, in _connection_for_bind\n engine, execution_options)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 403, in _connection_for_bind\n conn = bind.contextual_connect()\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2112, in contextual_connect\n self._wrap_pool_connect(self.pool.connect, None),\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2151, in _wrap_pool_connect\n e, dialect, self)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1461, in _handle_dbapi_exception_noconnection\n util.raise_from_cause(newraise, exc_info)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause\n reraise(type(exception), exception, tb=exc_tb, cause=cause)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2147, in _wrap_pool_connect\n return fn()\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 387, in connect\n return _ConnectionFairy._checkout(self)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 766, in _checkout\n fairy = _ConnectionRecord.checkout(pool)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 516, in checkout\n rec = pool._do_get()\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1138, in _do_get\n self._dec_overflow()\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__\n compat.reraise(exc_type, exc_value, exc_tb)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1135, in _do_get\n return self._create_connection()\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 333, in _create_connection\n return _ConnectionRecord(self)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 461, in __init__\n self.__connect(first_connect_check=True)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 651, in __connect\n connection = pool._invoke_creator(self)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 105, in connect\n return dialect.connect(*cargs, **cparams)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 393, in connect\n return self.dbapi.connect(*cargs, **cparams)\n', u' File "/usr/lib/python2.7/site-packages/pymysql/__init__.py", line 90, in Connect\n return Connection(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 706, in __init__\n self.connect()\n', u' File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 932, in connect\n self._request_authentication()\n', u' File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1152, in _request_authentication\n auth_packet = self._read_packet()\n', u' File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1014, in _read_packet\n packet.check_error()\n', u' File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 393, in check_error\n err.raise_mysql_exception(self._data)\n', u' File "/usr/lib/python2.7/site-packages/pymysql/err.py", line 107, in raise_mysql_exception\n raise errorclass(errno, errval)\n', u"OperationalError: (pymysql.err.OperationalError) (1040, u'Too many connections')\n"]. 2018-01-30 06:57:52.850 3670 WARNING nova.scheduler.utils [req-6f3db77c-6b4b-4525-b8d2-702dedb626b3 9ca3661a7c594f1ba7848ef69ce2ca04 9e1cb8200d164336be675deeaad131b3 - - -] [instance: 8150bb45-7758-4112-8593-e602b33ab4a5] Setting instance to ERROR state.: RemoteError: Remote error: OperationalError (pymysql.err.OperationalError) (1040, u'Too many connections') ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1749167 Title: nova show can not get an instance information, and this instance can be queried from nova list. Status in OpenStack Compute (nova): New Bug description: Description =========== nova delete raise "NO server with a name or ID of 8150bb45-7758-4112-8593-e602b33ab4a5 exists", but this instance is queried from "nova list" command. Steps to reproduce ================== A chronological list of steps which will bring off the issue you noticed: * I did create some instances, instance with uuid 8150bb45-7758-4112-8593-e602b33ab4a5 is scheduling all the time. nova list --all | grep scheduling | 8150bb45-7758-4112-8593-e602b33ab4a5 | test_aa | BUILD | scheduling | NOSTATE | * then I did delete this instance: nova delete 8150bb45-7758-4112-8593-e602b33ab4a5 NO server with a name or ID of 8150bb45-7758-4112-8593-e602b33ab4a5 exists Error(commandError): unable to delete this specified server(s). Expected result =============== Delete operation can success. Actual result ============= Delete operation failed. Instance information in Database is inconsistent; I look at database, instance's cell is None, can't see this instance in build_request table. And this instance's detail information was found in cell0 database. Environment =========== 1. Exact version of OpenStack you are running. See the following list for all releases: http://docs.openstack.org/releases/ Pike Logs & Configs ============== [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming\n res = self.dispatcher.dispatch(message)\n', u' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', u' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch\n result = func(ctxt, **new_args)\n', u' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 232, in inner\n return func(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 153, in select_destinations\n alloc_reqs_by_rp_uuid, provider_summaries)\n', u' File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 89, in select_destinations\n alloc_reqs_by_rp_uuid, provider_summaries)\n', u' File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 163, in _schedule\n provider_summaries)\n', u' File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 344, in _get_all_host_states\n spec_obj)\n', u' File "/usr/lib/python2.7/site-packages/nova/scheduler/host_manager.py", line 659, in get_host_states_by_uuids\n context, cells, compute_uuids=compute_uuids)\n', u' File "/usr/lib/python2.7/site-packages/nova/scheduler/host_manager.py", line 627, in _get_computes_for_cells\n cctxt, compute_uuids))\n', u' File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 184, in wrapper\n result = fn(cls, context, *args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/nova/objects/compute_node.py", line 460, in get_all_by_uuids\n compute_uuids)\n', u' File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 235, in wrapper\n return f(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/nova/objects/compute_node.py", line 454, in _db_compute_node_get_all_by_uuids\n models.ComputeNode.uuid.in_(compute_uuids)).all()\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2703, in all\n return list(self)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2855, in __iter__\n return self._execute_and_instances(context)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2876, in _execute_and_instances\n close_with_result=True)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2885, in _get_bind_args\n **kw\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2867, in _connection_from_session\n conn = self.session.connection(**kw)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 998, in connection\n execution_options=execution_options)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1003, in _connection_for_bind\n engine, execution_options)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 403, in _connection_for_bind\n conn = bind.contextual_connect()\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2112, in contextual_connect\n self._wrap_pool_connect(self.pool.connect, None),\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2151, in _wrap_pool_connect\n e, dialect, self)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1461, in _handle_dbapi_exception_noconnection\n util.raise_from_cause(newraise, exc_info)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause\n reraise(type(exception), exception, tb=exc_tb, cause=cause)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2147, in _wrap_pool_connect\n return fn()\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 387, in connect\n return _ConnectionFairy._checkout(self)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 766, in _checkout\n fairy = _ConnectionRecord.checkout(pool)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 516, in checkout\n rec = pool._do_get()\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1138, in _do_get\n self._dec_overflow()\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__\n compat.reraise(exc_type, exc_value, exc_tb)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1135, in _do_get\n return self._create_connection()\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 333, in _create_connection\n return _ConnectionRecord(self)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 461, in __init__\n self.__connect(first_connect_check=True)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 651, in __connect\n connection = pool._invoke_creator(self)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 105, in connect\n return dialect.connect(*cargs, **cparams)\n', u' File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 393, in connect\n return self.dbapi.connect(*cargs, **cparams)\n', u' File "/usr/lib/python2.7/site-packages/pymysql/__init__.py", line 90, in Connect\n return Connection(*args, **kwargs)\n', u' File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 706, in __init__\n self.connect()\n', u' File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 932, in connect\n self._request_authentication()\n', u' File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1152, in _request_authentication\n auth_packet = self._read_packet()\n', u' File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1014, in _read_packet\n packet.check_error()\n', u' File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 393, in check_error\n err.raise_mysql_exception(self._data)\n', u' File "/usr/lib/python2.7/site-packages/pymysql/err.py", line 107, in raise_mysql_exception\n raise errorclass(errno, errval)\n', u"OperationalError: (pymysql.err.OperationalError) (1040, u'Too many connections')\n"]. 2018-01-30 06:57:52.850 3670 WARNING nova.scheduler.utils [req-6f3db77c-6b4b-4525-b8d2-702dedb626b3 9ca3661a7c594f1ba7848ef69ce2ca04 9e1cb8200d164336be675deeaad131b3 - - -] [instance: 8150bb45-7758-4112-8593-e602b33ab4a5] Setting instance to ERROR state.: RemoteError: Remote error: OperationalError (pymysql.err.OperationalError) (1040, u'Too many connections') To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1749167/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
