** Also affects: nova/queens
   Importance: Undecided
       Status: New

** Also affects: nova/pike
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1745977

Title:
  When source compute service up, will not destroy and clean up those
  instances which be evacuated then be deleted.

Status in OpenStack Compute (nova):
  In Progress
Status in OpenStack Compute (nova) pike series:
  New
Status in OpenStack Compute (nova) queens series:
  New

Bug description:
  Description
  ===========
  When the instance evacuated to destination host successfully, then deleted 
this instance. The source host up will cleanup this instance failed.

  Steps to reproduce
  ==================
  1.deploy a local instance in source host.
  2.power off the source host.
  3.evacuate the instance to destination host.
  4.delete this instance.
  5.power on the source host.

  Expected result
  ===============
  The source host nova-compute service cleanup this evacuated and deleted 
instance.

  Actual result
  =============
  This instance still on source host.

  Environment
  ===========
  Openstack Pike
  Libvirt + KVM
  ovs network

  
  Logs & Configs
  ==============
  source host nova-compute log:

  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service 
[req-7bdfe28f-0464-4af8-bdd0-2d433b25d84a - - - - -] Error starting thread.: 
InstanceNotFound_Remote: Instance 19022200-7abc-423d-90bd-e9dcd0887679 could 
not be found.
  Traceback (most recent call last):

    File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 
125, in _object_dispatch
      return getattr(target, method)(*args, **kwargs)

    File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 
184, in wrapper
      result = fn(cls, context, *args, **kwargs)

    File "/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 474, 
in get_by_uuid
      use_slave=use_slave)

    File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 
235, in wrapper
      return f(*args, **kwargs)

    File "/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 466, 
in _db_instance_get_by_uuid
      columns_to_join=columns_to_join)

    File "/usr/lib/python2.7/site-packages/nova/db/api.py", line 744, in 
instance_get_by_uuid
      return IMPL.instance_get_by_uuid(context, uuid, columns_to_join)

    File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 
179, in wrapper
      return f(*args, **kwargs)

    File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 
280, in wrapped
      return f(context, *args, **kwargs)

    File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 
1911, in instance_get_by_uuid
      columns_to_join=columns_to_join)

    File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 
1920, in _instance_get_by_uuid
      raise exception.InstanceNotFound(instance_id=uuid)

  InstanceNotFound: Instance 19022200-7abc-423d-90bd-e9dcd0887679 could not be 
found.
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service Traceback (most 
recent call last):
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/oslo_service/service.py", line 721, in 
run_service
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     service.start()
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/nova/service.py", line 156, in start
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     
self.manager.init_host()
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1173, in 
init_host
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     
self._destroy_evacuated_instances(context)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 691, in 
_destroy_evacuated_instances
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     bdi, 
destroy_disks)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 909, in 
destroy
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     destroy_disks)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1032, in 
cleanup
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     attempts = 
int(instance.system_metadata.get('clean_attempts',
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 67, in 
getter
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     
self.obj_load_attr(name)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 1131, in 
obj_load_attr
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     
self._load_generic(attrname)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 858, in 
_load_generic
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     
expected_attrs=[attrname])
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 177, in 
wrapper
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     args, kwargs)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/nova/conductor/rpcapi.py", line 240, in 
object_class_action_versions
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     args=args, 
kwargs=kwargs)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 167, in 
call
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     retry=self.retry)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 123, in 
_send
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     timeout=timeout, 
retry=retry)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 
566, in send
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     retry=retry)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 
557, in _send
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     raise result
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service 
InstanceNotFound_Remote: Instance 19022200-7abc-423d-90bd-e9dcd0887679 could 
not be found.
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service Traceback (most 
recent call last):
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 125, in 
_object_dispatch
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     return 
getattr(target, method)(*args, **kwargs)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 184, in 
wrapper
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     result = fn(cls, 
context, *args, **kwargs)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 474, in 
get_by_uuid
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     
use_slave=use_slave)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 235, in 
wrapper
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     return f(*args, 
**kwargs)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 466, in 
_db_instance_get_by_uuid
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     
columns_to_join=columns_to_join)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/nova/db/api.py", line 744, in 
instance_get_by_uuid
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     return 
IMPL.instance_get_by_uuid(context, uuid, columns_to_join)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 179, in 
wrapper
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     return f(*args, 
**kwargs)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 280, in 
wrapped
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     return f(context, 
*args, **kwargs)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 1911, in 
instance_get_by_uuid
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     
columns_to_join=columns_to_join)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service   File 
"/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 1920, in 
_instance_get_by_uuid
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service     raise 
exception.InstanceNotFound(instance_id=uuid)
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service
  2018-01-29 10:28:48.664 9364 ERROR oslo_service.service InstanceNotFound: 
Instance 19022200-7abc-423d-90bd-e9dcd0887679 could not be found.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1745977/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to     : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

Reply via email to