*** This bug is a duplicate of bug 1794996 ***
https://bugs.launchpad.net/bugs/1794996
This might already be fixed for bug 1794996:
https://review.openstack.org/#/q/I1f4b3540dd453650f94333b36d7504ba164192f7
But that was not backported to stable/ocata.
If that fixes your issue we could work on backporting
https://review.openstack.org/#/q/topic:bug/1550919+branch:stable/pike to
stable/ocata.
** This bug has been marked a duplicate of bug 1794996
_destroy_evacuated_instances fails and kills n-cpu startup if lazy-loading
flavor on a deleted instance
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1817670
Title:
nova-compute could not destroy evacuated instance which was deleted
Status in OpenStack Compute (nova):
New
Bug description:
Description
===========
After evacuation, nova-compute where VMs were evacuated could not restart
emitting InstanceNotFound exception when init_host() called.
Steps to reproduce
==================
0) service nova-compute stop at 'compute01'
1) nova evacuate 'compute01'
2) nova delete 'server at compute01'
3) service nova-compute restart at 'compute01'
Expected result
===============
nova-compute normally started
Actual result
=============
nova-compute could not started
Environment
===========
1. Exact version of OpenStack you are running. See the following
list for all releases: http://docs.openstack.org/releases/
stable/ocata
2. Which hypervisor did you use?
(For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...)
What's the version of that?
libvirt + kvm
2. Which storage type did you use?
(For example: Ceph, LVM, GPFS, ...)
What's the version of that?
Ceph
3. Which networking type did you use?
(For example: nova-network, Neutron with OpenVSwitch, ...)
neutron with linux bridge
Logs & Configs
==============
2019-02-26 16:24:29.015 8901 INFO nova.virt.libvirt.driver
[req-dc2a5061-29aa-4097-9a3e-83de36804460 - - - - -] [instance:
25ab917a-1506-4154-9902-5b8b550ec929] Deleting instance files
/var/lib/nova/instances/25ab917a-1506-4154-9902-5b8b550ec929_del
2019-02-26 16:24:29.017 8901 INFO nova.virt.libvirt.driver
[req-dc2a5061-29aa-4097-9a3e-83de36804460 - - - - -] [instance:
25ab917a-1506-4154-9902-5b8b550ec929] Deletion of
/var/lib/nova/instances/25ab917a-1506-4154-9902-5b8b550ec929_del complete
2019-02-26 16:24:29.020 8901 DEBUG oslo_messaging._drivers.amqpdriver
[req-dc2a5061-29aa-4097-9a3e-83de36804460 - - - - -] CALL msg_id:
9bbf4371e977450d98a519dac213592b exchange 'nova' topic 'conductor' _send
/opt/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:550
2019-02-26 16:24:29.085 8901 DEBUG oslo_messaging._drivers.amqpdriver [-]
received reply msg_id: 9bbf4371e977450d98a519dac213592b __call__
/opt/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:407
2019-02-26 16:24:29.093 8901 DEBUG oslo_messaging._drivers.amqpdriver
[req-dc2a5061-29aa-4097-9a3e-83de36804460 - - - - -] CAST unique_id:
d5a0d3b93fbf42b6a379b9507e9c2a9a FANOUT topic 'scheduler' _send
/opt/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:539
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
[req-dc2a5061-29aa-4097-9a3e-83de36804460 - - - - -] Error starting thread.
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service Traceback (most
recent call last):
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/lib/python2.7/site-packages/oslo_service/service.py", line 722,
in run_service
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service service.start()
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/src/nova/nova/service.py", line 144, in start
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
self.manager.init_host()
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/src/nova/nova/compute/manager.py", line 1155, in init_host
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
self._destroy_evacuated_instances(context)
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/src/nova/nova/compute/manager.py", line 674, in
_destroy_evacuated_instances
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service bdi,
destroy_disks)
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/src/nova/nova/virt/libvirt/driver.py", line 915, in destroy
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service destroy_disks,
migrate_data)
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/src/nova/nova/virt/libvirt/driver.py", line 1034, in cleanup
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service instance.save()
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/lib/python2.7/site-packages/oslo_versionedobjects/base.py",
line 210, in wrapper
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service ctxt, self,
fn.__name__, args, kwargs)
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/src/nova/nova/conductor/rpcapi.py", line 244, in object_action
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
objmethod=objmethod, args=args, kwargs=kwargs)
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line
169, in call
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service retry=self.retry)
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/lib/python2.7/site-packages/oslo_messaging/transport.py", line
97, in _send
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service timeout=timeout,
retry=retry)
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py",
line 566, in send
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service retry=retry)
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py",
line 557, in _send
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service raise result
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
InstanceNotFound_Remote: Instance 25ab917a-1506-4154-9902-5b8b550ec929 could
not be found.
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service Traceback (most
recent call last):
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/src/nova/nova/conductor/manager.py", line 93, in
_object_dispatch
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service return
getattr(target, method)(*args, **kwargs)
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/lib/python2.7/site-packages/oslo_versionedobjects/base.py",
line 226, in wrapper
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service return fn(self,
*args, **kwargs)
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/src/nova/nova/objects/instance.py", line 792, in save
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
columns_to_join=_expected_cols(expected_attrs))
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/src/nova/nova/db/api.py", line 840, in
instance_update_and_get_original
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service expected=expected)
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/src/nova/nova/db/sqlalchemy/api.py", line 180, in wrapper
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service return f(*args,
**kwargs)
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/lib/python2.7/site-packages/oslo_db/api.py", line 151, in
wrapper
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service ectxt.value =
e.inner_exc
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220,
in __exit__
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196,
in force_reraise
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/lib/python2.7/site-packages/oslo_db/api.py", line 139, in
wrapper
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service return f(*args,
**kwargs)
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/src/nova/nova/db/sqlalchemy/api.py", line 251, in wrapped
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service return f(context,
*args, **kwargs)
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/src/nova/nova/db/sqlalchemy/api.py", line 2714, in
instance_update_and_get_original
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
columns_to_join=columns_to_join)
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service File
"/opt/openstack/src/nova/nova/db/sqlalchemy/api.py", line 1910, in
_instance_get_by_uuid
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service raise
exception.InstanceNotFound(instance_id=uuid)
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service InstanceNotFound:
Instance 25ab917a-1506-4154-9902-5b8b550ec929 could not be found.
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
2019-02-26 16:24:29.097 8901 ERROR oslo_service.service
2019-02-26 16:24:29.102 8901 DEBUG oslo_concurrency.lockutils
[req-67496f45-7809-41c4-a6a1-365574533016 - - - - -] Acquired semaphore
"singleton_lock" lock
/opt/openstack/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212
2019-02-26 16:24:29.103 8901 DEBUG oslo_concurrency.lockutils
[req-67496f45-7809-41c4-a6a1-365574533016 - - - - -] Releasing semaphore
"singleton_lock" lock
/opt/openstack/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1817670/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
