This doesn't seem like a nova problem - there is a problem with the DB connection configuration (maybe not scaled properly for your environment?).
2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi OperationalError: (pymysql.err.OperationalError) (1040, u'Too many connections') (Background on this error at: http://sqlalche.me/e/e3q8) ** Changed in: nova Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1790027 Title: Error retrieving instances list Status in OpenStack Compute (nova): Invalid Bug description: Queens release (After restart works ok) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi [req-73c5fdb2-822a-416b-b91c-b7e6fa36fb67 a700160233594eddb2621493b1a1d8d2 08847efa215b4fa6bbe6d1437e38194b - default default] Unexpected exception in API method: OperationalError: (pymysql.err.OperationalError) (1040, u'Too many connections') (Background on this error at: http://sqlalche.me/e/e3q8) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi Traceback (most recent call last): 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 788, in wrapped 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi return f(*args, **kwargs) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/servers.py", line 439, in show 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id, is_detail=True) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/servers.py", line 344, in _get_server 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi expected_attrs=expected_attrs) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/openstack/common.py", line 479, in get_instance 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi expected_attrs=expected_attrs) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 2348, in get 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi expected_attrs) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 2308, in _get_instance 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi context, instance_uuid, expected_attrs=expected_attrs) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 184, in wrapper 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 475, in get_by_uuid 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi use_slave=use_slave) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 225, in wrapper 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi return f(*args, **kwargs) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 467, in _db_instance_get_by_uuid 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi columns_to_join=columns_to_join) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/db/api.py", line 764, in instance_get_by_uuid 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi return IMPL.instance_get_by_uuid(context, uuid, columns_to_join) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 169, in wrapper 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi return f(*args, **kwargs) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 270, in wrapped 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi return f(context, *args, **kwargs) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 1872, in instance_get_by_uuid 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi columns_to_join=columns_to_join) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 1877, in _instance_get_by_uuid 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi filter_by(uuid=uuid).\ 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2778, in first 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi ret = list(self[0:1]) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2570, in __getitem__ 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi return list(res) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2878, in __iter__ 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi return self._execute_and_instances(context) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2899, in _execute_and_instances 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi close_with_result=True) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2908, in _get_bind_args 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi **kw 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2890, in _connection_from_session 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi conn = self.session.connection(**kw) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1035, in connection 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi execution_options=execution_options) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1040, in _connection_for_bind 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi engine, execution_options) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 409, in _connection_for_bind 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi conn = bind.contextual_connect() 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2123, in contextual_connect 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi self._wrap_pool_connect(self.pool.connect, None), 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2162, in _wrap_pool_connect 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi e, dialect, self) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1472, in _handle_dbapi_exception_noconnection 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi util.raise_from_cause(newraise, exc_info) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi reraise(type(exception), exception, tb=exc_tb, cause=cause) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2158, in _wrap_pool_connect 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi return fn() 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 403, in connect 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi return _ConnectionFairy._checkout(self) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 782, in _checkout 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi fairy = _ConnectionRecord.checkout(pool) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 532, in checkout 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi rec = pool._do_get() 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1186, in _do_get 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi self._dec_overflow() 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__ 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi compat.reraise(exc_type, exc_value, exc_tb) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1183, in _do_get 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi return self._create_connection() 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 350, in _create_connection 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi return _ConnectionRecord(self) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 477, in __init__ 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi self.__connect(first_connect_check=True) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 667, in __connect 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi connection = pool._invoke_creator(self) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 105, in connect 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi return dialect.connect(*cargs, **cparams) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 410, in connect 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi return self.dbapi.connect(*cargs, **cparams) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/pymysql/__init__.py", line 90, in Connect 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi return Connection(*args, **kwargs) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 706, in __init__ 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi self.connect() 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 932, in connect 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi self._request_authentication() 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1152, in _request_authentication 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi auth_packet = self._read_packet() 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1014, in _read_packet 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi packet.check_error() 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 393, in check_error 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/pymysql/err.py", line 107, in raise_mysql_exception 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi OperationalError: (pymysql.err.OperationalError) (1040, u'Too many connections') (Background on this error at: http://sqlalche.me/e/e3q8) 2018-08-31 04:04:11.328 58009 ERROR nova.api.openstack.wsgi 2018-08-31 04:04:11.339 58009 INFO nova.api.openstack.wsgi [req-73c5fdb2-822a-416b-b91c-b7e6fa36fb67 a700160233594eddb2621493b1a1d8d2 08847efa215b4fa6bbe6d1437e38194b - default default] HTTP exception thrown: Неожиданная ошибка API. Сообщите об этом в http://bugs.launchpad.net/nova/ и прикрепите протокол API Nova, если возможно. <class 'sqlalchemy.exc.OperationalError'> To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1790027/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
