This means that the command could not connect to the nova cell DB. As
you can see from the logs there was a time out waiting for connection to
the cells due to which did_not_respond_sentinels were attached to the
cell results
(https://github.com/openstack/nova/blob/master/nova/context.py#L464) .
That is why it fails. Please check your cell DB connection and see if
you are able to connect to it.
** Changed in: nova
Status: New => Invalid
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1770175
Title:
When list the servers, report : 'TypeError: 'object' object is not
iterable'
Status in OpenStack Compute (nova):
Invalid
Bug description:
Description
===========
When do the query of the servers(by CLI or Dashboard), sometimes report
errors like the follow:
[root@ctrl01 ~]# openstack server list
Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/
and attach the Nova API log if possible.
<type 'exceptions.TypeError'> (HTTP 500) (Request-ID:
req-f7e1bfb0-926c-43ca-8fbd-66bcfe15e409)
Steps to reproduce
==================
(See Description)
Expected result
===============
Get the correct result(the vm server list)
Actual result
=============
Environment
===========
[root@ctrl01 ~]# rpm -qa | grep nova
python2-novaclient-9.1.1-1.el7.noarch
openstack-nova-novncproxy-17.0.2-1.el7.noarch
openstack-nova-conductor-17.0.2-1.el7.noarch
openstack-nova-common-17.0.2-1.el7.noarch
openstack-nova-console-17.0.2-1.el7.noarch
openstack-nova-placement-api-17.0.2-1.el7.noarch
openstack-nova-scheduler-17.0.2-1.el7.noarch
python-nova-17.0.2-1.el7.noarch
openstack-nova-api-17.0.2-1.el7.noarch
Logs & Configs
==============
2018-05-09 22:08:47.085 10667 DEBUG nova.osapi_compute.wsgi.server
[req-483c7fc7-8094-4a73-83f4-f8e0d39deaef - - - - -] (10667) accepted
('172.16.2.240', 40984) server
/usr/lib/python2.7/site-packages/eventlet/wsgi.py:883
2018-05-09 22:08:47.173 10668 INFO nova.metadata.wsgi.server
[req-483c7fc7-8094-4a73-83f4-f8e0d39deaef - - - - -] (10668) wsgi starting up
on http://0.0.0.0:8775
2018-05-09 22:08:47.235 10669 INFO nova.metadata.wsgi.server
[req-483c7fc7-8094-4a73-83f4-f8e0d39deaef - - - - -] (10669) wsgi starting up
on http://0.0.0.0:8775
2018-05-09 22:09:48.275 10667 DEBUG nova.api.openstack.wsgi
[req-f7e1bfb0-926c-43ca-8fbd-66bcfe15e409 96ce2e706922453aa600078b866f8a68
ea02f765bd584e8c813459cb42063f76 - default default] Calling method '<bound
method ServersController.detail of
<nova.api.openstack.compute.servers.ServersController object at 0x8ad4290>>'
_process_stack /usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:607
2018-05-09 22:09:48.279 10667 DEBUG nova.compute.api
[req-f7e1bfb0-926c-43ca-8fbd-66bcfe15e409 96ce2e706922453aa600078b866f8a68
ea02f765bd584e8c813459cb42063f76 - default default] Searching by: {'deleted':
False, 'project_id': u'ea02f765bd584e8c813459cb42063f76'} get_all
/usr/lib/python2.7/site-packages/nova/compute/api.py:2347
2018-05-09 22:10:48.301 10667 DEBUG oslo_db.sqlalchemy.engines
[req-f7e1bfb0-926c-43ca-8fbd-66bcfe15e409 96ce2e706922453aa600078b866f8a68
ea02f765bd584e8c813459cb42063f76 - default default] MySQL server mode set to
STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
_check_effective_sql_mode
/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:290
2018-05-09 22:10:48.321 10667 DEBUG nova.context
[req-f7e1bfb0-926c-43ca-8fbd-66bcfe15e409 96ce2e706922453aa600078b866f8a68
ea02f765bd584e8c813459cb42063f76 - default default] Found 2 cells:
00000000-0000-0000-0000-000000000000(cell0),d34a74ee-bc98-4f36-882e-27b2acb2e510(cell1)
load_cells /usr/lib/python2.7/site-packages/nova/context.py:474
2018-05-09 22:10:48.362 10667 DEBUG oslo_concurrency.lockutils
[req-f7e1bfb0-926c-43ca-8fbd-66bcfe15e409 96ce2e706922453aa600078b866f8a68
ea02f765bd584e8c813459cb42063f76 - default default] Lock
"00000000-0000-0000-0000-000000000000" acquired by
"nova.context.get_or_set_cached_cell_and_set_connections" :: waited 0.000s
inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:273
2018-05-09 22:10:48.365 10667 DEBUG oslo_concurrency.lockutils
[req-f7e1bfb0-926c-43ca-8fbd-66bcfe15e409 96ce2e706922453aa600078b866f8a68
ea02f765bd584e8c813459cb42063f76 - default default] Lock
"00000000-0000-0000-0000-000000000000" released by
"nova.context.get_or_set_cached_cell_and_set_connections" :: held 0.003s inner
/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:285
2018-05-09 22:10:48.369 10667 DEBUG oslo_concurrency.lockutils
[req-f7e1bfb0-926c-43ca-8fbd-66bcfe15e409 96ce2e706922453aa600078b866f8a68
ea02f765bd584e8c813459cb42063f76 - default default] Lock
"d34a74ee-bc98-4f36-882e-27b2acb2e510" acquired by
"nova.context.get_or_set_cached_cell_and_set_connections" :: waited 0.000s
inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:273
2018-05-09 22:10:48.374 10667 DEBUG oslo_concurrency.lockutils
[req-f7e1bfb0-926c-43ca-8fbd-66bcfe15e409 96ce2e706922453aa600078b866f8a68
ea02f765bd584e8c813459cb42063f76 - default default] Lock
"d34a74ee-bc98-4f36-882e-27b2acb2e510" released by
"nova.context.get_or_set_cached_cell_and_set_connections" :: held 0.005s inner
/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:285
2018-05-09 22:11:48.326 10667 WARNING nova.context
[req-f7e1bfb0-926c-43ca-8fbd-66bcfe15e409 96ce2e706922453aa600078b866f8a68
ea02f765bd584e8c813459cb42063f76 - default default] Timed out waiting for
response from cell 00000000-0000-0000-0000-000000000000: CellTimeout: Timeout
waiting for response from cell
2018-05-09 22:11:48.327 10667 WARNING nova.context
[req-f7e1bfb0-926c-43ca-8fbd-66bcfe15e409 96ce2e706922453aa600078b866f8a68
ea02f765bd584e8c813459cb42063f76 - default default] Timed out waiting for
response from cell d34a74ee-bc98-4f36-882e-27b2acb2e510: CellTimeout: Timeout
waiting for response from cell
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi
[req-f7e1bfb0-926c-43ca-8fbd-66bcfe15e409 96ce2e706922453aa600078b866f8a68
ea02f765bd584e8c813459cb42063f76 - default default] Unexpected exception in API
method: TypeError: 'object' object is not iterable
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi Traceback (most
recent call last):
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi File
"/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 788, in
wrapped
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi return
f(*args, **kwargs)
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi File
"/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 181,
in wrapper
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi return
func(*args, **kwargs)
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi File
"/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 181,
in wrapper
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi return
func(*args, **kwargs)
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi File
"/usr/lib/python2.7/site-packages/nova/api/openstack/compute/servers.py", line
165, in detail
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi servers =
self._get_servers(req, is_detail=True)
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi File
"/usr/lib/python2.7/site-packages/nova/api/openstack/compute/servers.py", line
308, in _get_servers
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi
sort_keys=sort_keys, sort_dirs=sort_dirs)
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi File
"/usr/lib/python2.7/site-packages/nova/compute/api.py", line 2462, in get_all
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi context,
filters, limit, marker, fields, sort_keys, sort_dirs)
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi File
"/usr/lib/python2.7/site-packages/nova/compute/instance_list.py", line 107, in
get_instance_objects_sorted
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi
expected_attrs)
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi File
"/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 1199, in
_make_instance_list
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi for db_inst
in db_inst_list:
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi File
"/usr/lib/python2.7/site-packages/nova/compute/multi_cell_list.py", line 259,
in get_records_sorted
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi for i in
heapq.merge(*results.values()):
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi File
"/usr/lib64/python2.7/heapq.py", line 372, in merge
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi for itnum, it
in enumerate(map(iter, iterables)):
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi TypeError:
'object' object is not iterable
2018-05-09 22:11:48.328 10667 ERROR nova.api.openstack.wsgi
2018-05-09 22:11:48.421 10667 INFO nova.api.openstack.wsgi
[req-f7e1bfb0-926c-43ca-8fbd-66bcfe15e409 96ce2e706922453aa600078b866f8a68
ea02f765bd584e8c813459cb42063f76 - default default] HTTP exception thrown:
Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and
attach the Nova API log if possible.
<type 'exceptions.TypeError'>
2018-05-09 22:11:48.422 10667 DEBUG nova.api.openstack.wsgi
[req-f7e1bfb0-926c-43ca-8fbd-66bcfe15e409 96ce2e706922453aa600078b866f8a68
ea02f765bd584e8c813459cb42063f76 - default default] Returning 500 to user:
Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and
attach the Nova API log if possible.
<type 'exceptions.TypeError'> __call__
/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:1064
2018-05-09 22:11:48.424 10667 INFO nova.osapi_compute.wsgi.server
[req-f7e1bfb0-926c-43ca-8fbd-66bcfe15e409 96ce2e706922453aa600078b866f8a68
ea02f765bd584e8c813459cb42063f76 - default default] 172.16.2.240 "GET
/v2.1/servers/detail HTTP/1.1" status: 500 len: 626 time: 181.3375800
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1770175/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
