Public bug reported:

http://logs.openstack.org/70/215170/1/check/gate-tempest-dsvm-
nova-v21-full/3fdc0d6/console.html#_2015-08-21_16_04_53_513

2015-08-21 16:04:53.514 | Captured traceback:
2015-08-21 16:04:53.514 | ~~~~~~~~~~~~~~~~~~~
2015-08-21 16:04:53.514 |     Traceback (most recent call last):
2015-08-21 16:04:53.514 |       File 
"tempest/api/compute/admin/test_servers.py", line 81, in 
test_list_servers_by_admin_with_all_tenants
2015-08-21 16:04:53.514 |         body = self.client.list_servers(detail=True, 
**params)
2015-08-21 16:04:53.514 |       File 
"tempest/services/compute/json/servers_client.py", line 159, in list_servers
2015-08-21 16:04:53.514 |         resp, body = self.get(url)
2015-08-21 16:04:53.514 |       File 
"/opt/stack/new/tempest/.tox/full/local/lib/python2.7/site-packages/tempest_lib/common/rest_client.py",
 line 271, in get
2015-08-21 16:04:53.514 |         return self.request('GET', url, 
extra_headers, headers)
2015-08-21 16:04:53.514 |       File 
"/opt/stack/new/tempest/.tox/full/local/lib/python2.7/site-packages/tempest_lib/common/rest_client.py",
 line 643, in request
2015-08-21 16:04:53.515 |         resp, resp_body)
2015-08-21 16:04:53.515 |       File 
"/opt/stack/new/tempest/.tox/full/local/lib/python2.7/site-packages/tempest_lib/common/rest_client.py",
 line 754, in _error_checker
2015-08-21 16:04:53.515 |         raise exceptions.ServerFault(resp_body, 
message=message)
2015-08-21 16:04:53.515 |     tempest_lib.exceptions.ServerFault: Got server 
fault
2015-08-21 16:04:53.515 |     Details: Unexpected API Error. Please report this 
at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
2015-08-21 16:04:53.515 |     <class 'nova.exception.InstanceNotFound'>

There is a trace in the n-api logs when trying to lazy-load a flavor on
an instance:

http://logs.openstack.org/70/215170/1/check/gate-tempest-dsvm-
nova-v21-full/3fdc0d6/logs/screen-n-api.txt.gz?level=TRACE#_2015-08-21_15_39_06_148

2015-08-21 15:39:06.148 ERROR nova.api.openstack.extensions 
[req-5eca1fa9-7948-4a3d-bc80-7e84441bb74e 
tempest-ServersAdminTestJSON-973647583 tempest-ServersAdminTestJSON-692147874] 
Unexpected exception in API method
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions Traceback 
(most recent call last):
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/api/openstack/extensions.py", line 478, in wrapped
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     return 
f(*args, **kwargs)
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/api/openstack/compute/servers.py", line 264, in detail
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     servers = 
self._get_servers(req, is_detail=True)
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/api/openstack/compute/servers.py", line 389, in 
_get_servers
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     response 
= self._view_builder.detail(req, instance_list)
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/api/openstack/compute/views/servers.py", line 126, in 
detail
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     return 
self._list_view(self.show, request, instances, coll_name)
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/api/openstack/compute/views/servers.py", line 138, in 
_list_view
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     
server_list = [func(request, server)["server"] for server in servers]
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/api/openstack/compute/views/servers.py", line 266, in 
show
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     "flavor": 
self._get_flavor(request, instance),
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/api/openstack/compute/views/servers.py", line 198, in 
_get_flavor
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     
instance_type = instance.get_flavor()
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/objects/instance.py", line 890, in get_flavor
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     return 
getattr(self, attr)
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 
65, in getter
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     
self.obj_load_attr(name)
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/objects/instance.py", line 880, in obj_load_attr
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     
self._load_flavor()
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/objects/instance.py", line 811, in _load_flavor
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     
expected_attrs=['flavor', 'system_metadata'])
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 
169, in wrapper
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     result = 
fn(cls, context, *args, **kwargs)
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/objects/instance.py", line 435, in get_by_uuid
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     
use_slave=use_slave)
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/db/api.py", line 644, in instance_get_by_uuid
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     
columns_to_join, use_slave=use_slave)
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 230, in wrapper
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     return 
f(*args, **kwargs)
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 1722, in 
instance_get_by_uuid
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     
columns_to_join=columns_to_join, use_slave=use_slave)
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 1734, in 
_instance_get_by_uuid
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     raise 
exception.InstanceNotFound(instance_id=uuid)
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions 
InstanceNotFound: Instance b4025d7a-c2ef-4234-a6ee-28edf966bfe7 could not be 
found.
2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions 

So between the time that we've retrieved the flavor from the database
and the time we try to lazy-load the flavor, the instance is deleted and
this fails.

We could pre-load the flavor in this case, or read_deleted='yes' when
we're lazy-loading a flavor.

** Affects: nova
     Importance: High
         Status: Confirmed


** Tags: api unified-objects

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1487570

Title:
  test_list_servers_by_admin_with_all_tenants fails with
  InstanceNotFound trying to lazy-load flavor

Status in OpenStack Compute (nova):
  Confirmed

Bug description:
  http://logs.openstack.org/70/215170/1/check/gate-tempest-dsvm-
  nova-v21-full/3fdc0d6/console.html#_2015-08-21_16_04_53_513

  2015-08-21 16:04:53.514 | Captured traceback:
  2015-08-21 16:04:53.514 | ~~~~~~~~~~~~~~~~~~~
  2015-08-21 16:04:53.514 |     Traceback (most recent call last):
  2015-08-21 16:04:53.514 |       File 
"tempest/api/compute/admin/test_servers.py", line 81, in 
test_list_servers_by_admin_with_all_tenants
  2015-08-21 16:04:53.514 |         body = 
self.client.list_servers(detail=True, **params)
  2015-08-21 16:04:53.514 |       File 
"tempest/services/compute/json/servers_client.py", line 159, in list_servers
  2015-08-21 16:04:53.514 |         resp, body = self.get(url)
  2015-08-21 16:04:53.514 |       File 
"/opt/stack/new/tempest/.tox/full/local/lib/python2.7/site-packages/tempest_lib/common/rest_client.py",
 line 271, in get
  2015-08-21 16:04:53.514 |         return self.request('GET', url, 
extra_headers, headers)
  2015-08-21 16:04:53.514 |       File 
"/opt/stack/new/tempest/.tox/full/local/lib/python2.7/site-packages/tempest_lib/common/rest_client.py",
 line 643, in request
  2015-08-21 16:04:53.515 |         resp, resp_body)
  2015-08-21 16:04:53.515 |       File 
"/opt/stack/new/tempest/.tox/full/local/lib/python2.7/site-packages/tempest_lib/common/rest_client.py",
 line 754, in _error_checker
  2015-08-21 16:04:53.515 |         raise exceptions.ServerFault(resp_body, 
message=message)
  2015-08-21 16:04:53.515 |     tempest_lib.exceptions.ServerFault: Got server 
fault
  2015-08-21 16:04:53.515 |     Details: Unexpected API Error. Please report 
this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  2015-08-21 16:04:53.515 |     <class 'nova.exception.InstanceNotFound'>

  There is a trace in the n-api logs when trying to lazy-load a flavor
  on an instance:

  http://logs.openstack.org/70/215170/1/check/gate-tempest-dsvm-
  
nova-v21-full/3fdc0d6/logs/screen-n-api.txt.gz?level=TRACE#_2015-08-21_15_39_06_148

  2015-08-21 15:39:06.148 ERROR nova.api.openstack.extensions 
[req-5eca1fa9-7948-4a3d-bc80-7e84441bb74e 
tempest-ServersAdminTestJSON-973647583 tempest-ServersAdminTestJSON-692147874] 
Unexpected exception in API method
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions Traceback 
(most recent call last):
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/api/openstack/extensions.py", line 478, in wrapped
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     return 
f(*args, **kwargs)
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/api/openstack/compute/servers.py", line 264, in detail
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     servers 
= self._get_servers(req, is_detail=True)
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/api/openstack/compute/servers.py", line 389, in 
_get_servers
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     
response = self._view_builder.detail(req, instance_list)
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/api/openstack/compute/views/servers.py", line 126, in 
detail
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     return 
self._list_view(self.show, request, instances, coll_name)
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/api/openstack/compute/views/servers.py", line 138, in 
_list_view
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     
server_list = [func(request, server)["server"] for server in servers]
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/api/openstack/compute/views/servers.py", line 266, in 
show
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     
"flavor": self._get_flavor(request, instance),
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/api/openstack/compute/views/servers.py", line 198, in 
_get_flavor
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     
instance_type = instance.get_flavor()
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/objects/instance.py", line 890, in get_flavor
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     return 
getattr(self, attr)
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 
65, in getter
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     
self.obj_load_attr(name)
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/objects/instance.py", line 880, in obj_load_attr
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     
self._load_flavor()
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/objects/instance.py", line 811, in _load_flavor
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     
expected_attrs=['flavor', 'system_metadata'])
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 
169, in wrapper
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     result 
= fn(cls, context, *args, **kwargs)
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/objects/instance.py", line 435, in get_by_uuid
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     
use_slave=use_slave)
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/db/api.py", line 644, in instance_get_by_uuid
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     
columns_to_join, use_slave=use_slave)
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 230, in wrapper
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     return 
f(*args, **kwargs)
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 1722, in 
instance_get_by_uuid
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     
columns_to_join=columns_to_join, use_slave=use_slave)
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions   File 
"/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 1734, in 
_instance_get_by_uuid
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions     raise 
exception.InstanceNotFound(instance_id=uuid)
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions 
InstanceNotFound: Instance b4025d7a-c2ef-4234-a6ee-28edf966bfe7 could not be 
found.
  2015-08-21 15:39:06.148 22838 ERROR nova.api.openstack.extensions 

  So between the time that we've retrieved the flavor from the database
  and the time we try to lazy-load the flavor, the instance is deleted
  and this fails.

  We could pre-load the flavor in this case, or read_deleted='yes' when
  we're lazy-loading a flavor.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1487570/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to     : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

Reply via email to