Public bug reported:
Description
===========
nova-compute fails to resize instance that was booted by creating a volume,
recognizing it as an ephemeral disk.
Steps to reproduce
==================
1. Create an instance by selecting boot source as 'image' and creating a new
volume
2. Resize the instance
3. Confirm the resizing
Expected result
===============
successful resize
Actual result
=============
It passes until resize confirmation without any error but it fails after the
resize confirmation is hit.
Environment
===========
1. Exact version of OpenStack you are running. See the following
dpkg -l | grep nova
ii nova-common 2:13.0.0-0ubuntu2
all OpenStack Compute - common files
ii nova-compute 2:13.0.0-0ubuntu2
all OpenStack Compute - compute node base
ii nova-compute-kvm 2:13.0.0-0ubuntu2
all OpenStack Compute - compute node (KVM)
ii nova-compute-libvirt 2:13.0.0-0ubuntu2
all OpenStack Compute - compute node libvirt support
ii python-nova 2:13.0.0-0ubuntu2
all OpenStack Compute Python libraries
ii python-novaclient 2:3.3.1-2
all client library for OpenStack Compute API - Python 2.7
2. Which hypervisor did you use?
Libvirt + KVM
dpkg -l | egrep 'libvirt|qemu|kvm'
ii ipxe-qemu 1.0.0+git-20150424.a25a16d-1ubuntu1
all PXE boot firmware - ROM images for qemu
ii libvirt-bin 1.3.1-1ubuntu10.1
amd64 programs for the libvirt library
ii libvirt0:amd64 1.3.1-1ubuntu10.1
amd64 library for interfacing with different virtualization systems
ii nova-compute-kvm 2:13.0.0-0ubuntu2
all OpenStack Compute - compute node (KVM)
ii nova-compute-libvirt 2:13.0.0-0ubuntu2
all OpenStack Compute - compute node libvirt support
ii python-libvirt 1.3.1-1ubuntu1
amd64 libvirt Python bindings
ii qemu-block-extra:amd64 1:2.5+dfsg-5ubuntu10.2
amd64 extra block backend modules for qemu-system and qemu-utils
ii qemu-slof 20151103+dfsg-1ubuntu1
all Slimline Open Firmware -- QEMU PowerPC version
ii qemu-system 1:2.5+dfsg-5ubuntu10.2
amd64 QEMU full system emulation binaries
ii qemu-system-arm 1:2.5+dfsg-5ubuntu10.2
amd64 QEMU full system emulation binaries (arm)
ii qemu-system-common 1:2.5+dfsg-5ubuntu10.2
amd64 QEMU full system emulation binaries (common files)
ii qemu-system-mips 1:2.5+dfsg-5ubuntu10.2
amd64 QEMU full system emulation binaries (mips)
ii qemu-system-misc 1:2.5+dfsg-5ubuntu10.2
amd64 QEMU full system emulation binaries (miscelaneous)
ii qemu-system-ppc 1:2.5+dfsg-5ubuntu10.2
amd64 QEMU full system emulation binaries (ppc)
ii qemu-system-sparc 1:2.5+dfsg-5ubuntu10.2
amd64 QEMU full system emulation binaries (sparc)
ii qemu-system-x86 1:2.5+dfsg-5ubuntu10.2
amd64 QEMU full system emulation binaries (x86)
ii qemu-utils 1:2.5+dfsg-5ubuntu10.2
amd64 QEMU utilities
2. Which storage type did you use?
Ceph
dpkg -l | egrep 'ceph'
ii ceph-common 10.2.2-1xenial
amd64 common utilities to mount and interact with a ceph storage cluster
ii libcephfs1 10.2.2-1xenial
amd64 Ceph distributed file system client library
ii python-cephfs 10.2.2-1xenial
amd64 Python libraries for the Ceph libcephfs library
3. Which networking type did you use?
Neutron with OpenVSwitch
# On the api node
dpkg -l | grep neutron
ii neutron-common 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - common
ii neutron-plugin-ml2 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - ML2 plugin
ii neutron-server 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - server
ii python-neutron 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - Python library
ii python-neutron-fwaas 1:8.0.0-0ubuntu1 all
Firewall-as-a-Service driver for OpenStack Neutron
ii python-neutron-lib 0.0.2-2 all
Neutron shared routines and utilities - Python 2.7
ii python-neutronclient 1:4.1.1-2 all
client API library for Neutron - Python 2.7
# On the network node
dpkg -l | grep neutron
ii neutron-common 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - common
ii neutron-dhcp-agent 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - DHCP agent
ii neutron-l3-agent 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - l3 agent
ii neutron-metadata-agent 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - metadata agent
ii neutron-openvswitch-agent 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - Open vSwitch plugin agent
ii python-neutron 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - Python library
ii python-neutron-fwaas 1:8.0.0-0ubuntu1 all
Firewall-as-a-Service driver for OpenStack Neutron
ii python-neutron-lib 0.0.2-2 all
Neutron shared routines and utilities - Python 2.7
ii python-neutronclient 1:4.1.1-2 all
client API library for Neutron - Python 2.7
Logs & Configs
==============
2016-07-05 15:40:32.701 99291 DEBUG oslo_concurrency.lockutils
[req-41fc2f7f-8c45-4c2d-ba7b-2144fcbfd757
8e06675285464030a9fd7badf4b0939cba85e018ab8a20ea8cb911222fef7771
04e7fe0cfb19418a9ec2eacfe1d334d5 - - -] Releasing semaphore
"refresh_cache-9debce4f-ee92-4982-815f-769380728951" lock
/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:228
2016-07-05 15:40:32.702 99291 DEBUG oslo_concurrency.processutils
[req-41fc2f7f-8c45-4c2d-ba7b-2144fcbfd757
8e06675285464030a9fd7badf4b0939cba85e018ab8a20ea8cb911222fef7771
04e7fe0cfb19418a9ec2eacfe1d334d5 - - -] Running cmd (subprocess): rm -rf
/var/lib/nova/instances/9debce4f-ee92-4982-815f-769380728951_resize execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:326
2016-07-05 15:40:32.728 99291 DEBUG oslo_concurrency.processutils
[req-41fc2f7f-8c45-4c2d-ba7b-2144fcbfd757
8e06675285464030a9fd7badf4b0939cba85e018ab8a20ea8cb911222fef7771
04e7fe0cfb19418a9ec2eacfe1d334d5 - - -] CMD "rm -rf
/var/lib/nova/instances/9debce4f-ee92-4982-815f-769380728951_resize" returned:
0 in 0.026s execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:356
2016-07-05 15:40:32.741 99291 DEBUG nova.virt.libvirt.storage.rbd_utils
[req-41fc2f7f-8c45-4c2d-ba7b-2144fcbfd757
8e06675285464030a9fd7badf4b0939cba85e018ab8a20ea8cb911222fef7771
04e7fe0cfb19418a9ec2eacfe1d334d5 - - -] rbd image
9debce4f-ee92-4982-815f-769380728951_disk does not exist __init__
/usr/lib/python2.7/dist-packages/nova/virt/libvirt/storage/rbd_utils.py:64
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager
[req-41fc2f7f-8c45-4c2d-ba7b-2144fcbfd757
8e06675285464030a9fd7badf4b0939cba85e018ab8a20ea8cb911222fef7771
04e7fe0cfb19418a9ec2eacfe1d334d5 - - -] [instance:
9debce4f-ee92-4982-815f-769380728951] Setting instance vm_state to ERROR
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] Traceback (most recent call last):
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6588, in
_error_out_instance_on_exception
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] yield
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3531, in
_confirm_resize
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] network_info)
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 7484, in
confirm_migration
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] self._cleanup_resize(instance,
network_info)
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1258, in
_cleanup_resize
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] ignore_errors=True)
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 941,
in remove_snap
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] return
self.driver.remove_snap(self.rbd_name, name, ignore_errors)
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/storage/rbd_utils.py", line
396, in remove_snap
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] with RBDVolumeProxy(self,
str(volume), pool=pool) as vol:
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/storage/rbd_utils.py", line
65, in __init__
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] driver._disconnect_from_rados(client,
ioctx)
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] self.force_reraise()
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] six.reraise(self.type_, self.value,
self.tb)
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/storage/rbd_utils.py", line
61, in __init__
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] read_only=read_only)
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File "rbd.pyx", line 1042, in
rbd.Image.__init__ (/tmp/buildd/ceph-10.2.2/src/build/rbd.c:9862)
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] ImageNotFound: error opening image
9debce4f-ee92-4982-815f-769380728951_disk at snapshot None
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951]
2016-07-05 15:40:32.811 99291 DEBUG oslo_concurrency.lockutils
[req-41fc2f7f-8c45-4c2d-ba7b-2144fcbfd757
8e06675285464030a9fd7badf4b0939cba85e018ab8a20ea8cb911222fef7771
04e7fe0cfb19418a9ec2eacfe1d334d5 - - -] Lock
"9debce4f-ee92-4982-815f-769380728951" released by
"nova.compute.manager.do_confirm_resize" :: held 0.507s inner
/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:285
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
[req-41fc2f7f-8c45-4c2d-ba7b-2144fcbfd757
8e06675285464030a9fd7badf4b0939cba85e018ab8a20ea8cb911222fef7771
04e7fe0cfb19418a9ec2eacfe1d334d5 - - -] Exception during message handling:
error opening image 9debce4f-ee92-4982-815f-769380728951_disk at snapshot None
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher Traceback
(most recent call last):
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 138,
in _dispatch_and_reply
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
incoming.message))
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 185,
in _dispatch
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher return
self._do_dispatch(endpoint, method, ctxt, args)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 127,
in _do_dispatch
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher result =
func(ctxt, **new_args)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/exception.py", line 110, in wrapped
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher payload)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
self.force_reraise()
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
six.reraise(self.type_, self.value, self.tb)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/exception.py", line 89, in wrapped
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher return
f(self, context, *args, **kw)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 409, in
decorated_function
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher return
function(self, context, *args, **kwargs)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 387, in
decorated_function
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
kwargs['instance'], e, sys.exc_info())
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
self.force_reraise()
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
six.reraise(self.type_, self.value, self.tb)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 375, in
decorated_function
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher return
function(self, context, *args, **kwargs)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3507, in
confirm_resize
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
do_confirm_resize(context, instance, migration.id)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 274, in
inner
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher return
f(*args, **kwargs)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3505, in
do_confirm_resize
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
migration=migration)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3531, in
_confirm_resize
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
network_info)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 7484, in
confirm_migration
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
self._cleanup_resize(instance, network_info)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1258, in
_cleanup_resize
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
ignore_errors=True)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 941,
in remove_snap
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher return
self.driver.remove_snap(self.rbd_name, name, ignore_errors)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/storage/rbd_utils.py", line
396, in remove_snap
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher with
RBDVolumeProxy(self, str(volume), pool=pool) as vol:
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/storage/rbd_utils.py", line
65, in __init__
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
driver._disconnect_from_rados(client, ioctx)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
self.force_reraise()
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
six.reraise(self.type_, self.value, self.tb)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/storage/rbd_utils.py", line
61, in __init__
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
read_only=read_only)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"rbd.pyx", line 1042, in rbd.Image.__init__
(/tmp/buildd/ceph-10.2.2/src/build/rbd.c:9862)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
ImageNotFound: error opening image 9debce4f-ee92-4982-815f-769380728951_disk at
snapshot None
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
openstack server show 9debce4f-ee92-4982-815f-769380728951
+--------------------------------------+------------------------------------------------------------------+
| Field | Value
|
+--------------------------------------+------------------------------------------------------------------+
| OS-DCF:diskConfig | AUTO
|
| OS-EXT-AZ:availability_zone | lab
|
| OS-EXT-SRV-ATTR:host | h12
|
| OS-EXT-SRV-ATTR:hypervisor_hostname | h12.example.com
|
| OS-EXT-SRV-ATTR:instance_name | instance-00000150
|
| OS-EXT-STS:power_state | 4
|
| OS-EXT-STS:task_state | None
|
| OS-EXT-STS:vm_state | stopped
|
| OS-SRV-USG:launched_at | 2016-07-05T06:40:11.000000
|
| OS-SRV-USG:terminated_at | None
|
| accessIPv4 |
|
| accessIPv6 |
|
| addresses | 1st-network=172.16.0.12
|
| config_drive |
|
| created | 2016-07-05T03:32:37Z
|
| flavor | m1.large (4)
|
| hostId |
90f85ee8c380c68c55b169ff1dc9917729638d0e744020641ace2700 |
| id | 9debce4f-ee92-4982-815f-769380728951
|
| image |
|
| key_name | desktop
|
| name | e1
|
| os-extended-volumes:volumes_attached | [{u'id':
u'90ea9f31-57b4-4162-8986-47065b23f407'}] |
| project_id | 04e7fe0cfb19418a9ec2eacfe1d334d5
|
| properties |
|
| security_groups | [{u'name': u'default'}]
|
| status | SHUTOFF
|
| updated | 2016-07-05T07:04:11Z
|
| user_id |
8e06675285464030a9fd7badf4b0939cba85e018ab8a20ea8cb911222fef7771 |
+--------------------------------------+------------------------------------------------------------------+
openstack volume show 90ea9f31-57b4-4162-8986-47065b23f407
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value
|
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attachments | [{u'server_id':
u'9debce4f-ee92-4982-815f-769380728951', u'attachment_id':
u'd119c454-65f4-4c7b-8135-3238a3e7a424', u'attached_at':
u'2016-07-05T03:33:37.127003', u'host_name': None, u'volume_id':
u'90ea9f31-57b4-4162-8986-47065b23f407', u'device': u'/dev/vda', u'id':
u'90ea9f31-57b4-4162-8986-47065b23f407'}]
|
| availability_zone | nova
|
| bootable | true
|
| consistencygroup_id | None
|
| created_at | 2016-07-05T03:32:38.353914
|
| description |
|
| encrypted | False
|
| id | 90ea9f31-57b4-4162-8986-47065b23f407
|
| migration_status | None
|
| multiattach | False
|
| name |
|
| os-vol-host-attr:host | mitaka-cinder@ceph#ceph
|
| os-vol-mig-status-attr:migstat | None
|
| os-vol-mig-status-attr:name_id | None
|
| os-vol-tenant-attr:tenant_id | 04e7fe0cfb19418a9ec2eacfe1d334d5
|
| properties | attached_mode='rw', readonly='False'
|
| replication_status | disabled
|
| size | 40
|
| snapshot_id | None
|
| source_volid | None
|
| status | in-use
|
| type | ceph
|
| updated_at | 2016-07-05T03:33:37.161179
|
| user_id |
8e06675285464030a9fd7badf4b0939cba85e018ab8a20ea8cb911222fef7771
|
| volume_image_metadata | {u'description': u'', u'os_require_quiesce':
u'yes', u'checksum': u'cb3d84ccc990821e537fbe5227b3dbc3',
u'hw_qemu_guest_agent': u'yes', u'disk_format': u'raw', u'image_name':
u'os2-2016062303', u'hw_scsi_model': u'virtio-scsi', u'image_id':
u'083aadf6-6a4c-45aa-aab6-66b6d2b2c2ba', u'min_ram': u'0', u'container_format':
u'bare', u'min_disk': u'3', u'hw_disk_bus': u'scsi', u'size': u'2705326080'} |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
# On the Ceph cluster
⟫ sudo rbd ls openstack-volume | grep 9debce4f-ee92-4982-815f-769380728951_disk
1 ⟫ sudo rbd ls openstack-volume | grep 90ea9f31-57b4-4162-8986-47065b23f407
volume-90ea9f31-57b4-4162-8986-47065b23f407
⟫ sudo rbd info -p openstack-volume volume-90ea9f31-57b4-4162-8986-47065b23f407
rbd image 'volume-90ea9f31-57b4-4162-8986-47065b23f407':
size 40960 MB in 10240 objects
order 22 (4096 kB objects)
block_name_prefix: rbd_data.b412b0238e1f29
format: 2
features: layering, exclusive-lock, object-map, fast-diff, deep-flatten
flags:
(The volume size has not been changed.)
As you can see above, nova-compute tries to do something on the
9debce4f-ee92-4982-815f-769380728951_disk which doesn't actually exist,
recognizing it as an ephemeral disk.
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1599067
Title:
[Mitaka] Fails to resize the instance booted by creating a volume,
recognizing it as an ephemeral disk
Status in OpenStack Compute (nova):
New
Bug description:
Description
===========
nova-compute fails to resize instance that was booted by creating a volume,
recognizing it as an ephemeral disk.
Steps to reproduce
==================
1. Create an instance by selecting boot source as 'image' and creating a new
volume
2. Resize the instance
3. Confirm the resizing
Expected result
===============
successful resize
Actual result
=============
It passes until resize confirmation without any error but it fails after the
resize confirmation is hit.
Environment
===========
1. Exact version of OpenStack you are running. See the following
dpkg -l | grep nova
ii nova-common 2:13.0.0-0ubuntu2
all OpenStack Compute - common files
ii nova-compute 2:13.0.0-0ubuntu2
all OpenStack Compute - compute node base
ii nova-compute-kvm 2:13.0.0-0ubuntu2
all OpenStack Compute - compute node (KVM)
ii nova-compute-libvirt 2:13.0.0-0ubuntu2
all OpenStack Compute - compute node libvirt support
ii python-nova 2:13.0.0-0ubuntu2
all OpenStack Compute Python libraries
ii python-novaclient 2:3.3.1-2
all client library for OpenStack Compute API - Python 2.7
2. Which hypervisor did you use?
Libvirt + KVM
dpkg -l | egrep 'libvirt|qemu|kvm'
ii ipxe-qemu 1.0.0+git-20150424.a25a16d-1ubuntu1
all PXE boot firmware - ROM images for qemu
ii libvirt-bin 1.3.1-1ubuntu10.1
amd64 programs for the libvirt library
ii libvirt0:amd64 1.3.1-1ubuntu10.1
amd64 library for interfacing with different virtualization systems
ii nova-compute-kvm 2:13.0.0-0ubuntu2
all OpenStack Compute - compute node (KVM)
ii nova-compute-libvirt 2:13.0.0-0ubuntu2
all OpenStack Compute - compute node libvirt support
ii python-libvirt 1.3.1-1ubuntu1
amd64 libvirt Python bindings
ii qemu-block-extra:amd64 1:2.5+dfsg-5ubuntu10.2
amd64 extra block backend modules for qemu-system and qemu-utils
ii qemu-slof 20151103+dfsg-1ubuntu1
all Slimline Open Firmware -- QEMU PowerPC version
ii qemu-system 1:2.5+dfsg-5ubuntu10.2
amd64 QEMU full system emulation binaries
ii qemu-system-arm 1:2.5+dfsg-5ubuntu10.2
amd64 QEMU full system emulation binaries (arm)
ii qemu-system-common 1:2.5+dfsg-5ubuntu10.2
amd64 QEMU full system emulation binaries (common files)
ii qemu-system-mips 1:2.5+dfsg-5ubuntu10.2
amd64 QEMU full system emulation binaries (mips)
ii qemu-system-misc 1:2.5+dfsg-5ubuntu10.2
amd64 QEMU full system emulation binaries (miscelaneous)
ii qemu-system-ppc 1:2.5+dfsg-5ubuntu10.2
amd64 QEMU full system emulation binaries (ppc)
ii qemu-system-sparc 1:2.5+dfsg-5ubuntu10.2
amd64 QEMU full system emulation binaries (sparc)
ii qemu-system-x86 1:2.5+dfsg-5ubuntu10.2
amd64 QEMU full system emulation binaries (x86)
ii qemu-utils 1:2.5+dfsg-5ubuntu10.2
amd64 QEMU utilities
2. Which storage type did you use?
Ceph
dpkg -l | egrep 'ceph'
ii ceph-common 10.2.2-1xenial
amd64 common utilities to mount and interact with a ceph storage cluster
ii libcephfs1 10.2.2-1xenial
amd64 Ceph distributed file system client library
ii python-cephfs 10.2.2-1xenial
amd64 Python libraries for the Ceph libcephfs library
3. Which networking type did you use?
Neutron with OpenVSwitch
# On the api node
dpkg -l | grep neutron
ii neutron-common 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - common
ii neutron-plugin-ml2 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - ML2 plugin
ii neutron-server 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - server
ii python-neutron 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - Python library
ii python-neutron-fwaas 1:8.0.0-0ubuntu1 all
Firewall-as-a-Service driver for OpenStack Neutron
ii python-neutron-lib 0.0.2-2 all
Neutron shared routines and utilities - Python 2.7
ii python-neutronclient 1:4.1.1-2 all
client API library for Neutron - Python 2.7
# On the network node
dpkg -l | grep neutron
ii neutron-common 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - common
ii neutron-dhcp-agent 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - DHCP agent
ii neutron-l3-agent 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - l3 agent
ii neutron-metadata-agent 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - metadata agent
ii neutron-openvswitch-agent 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - Open vSwitch plugin
agent
ii python-neutron 2:8.1.0-0ubuntu0.16.04.2 all
Neutron is a virtual network service for Openstack - Python library
ii python-neutron-fwaas 1:8.0.0-0ubuntu1 all
Firewall-as-a-Service driver for OpenStack Neutron
ii python-neutron-lib 0.0.2-2 all
Neutron shared routines and utilities - Python 2.7
ii python-neutronclient 1:4.1.1-2 all
client API library for Neutron - Python 2.7
Logs & Configs
==============
2016-07-05 15:40:32.701 99291 DEBUG oslo_concurrency.lockutils
[req-41fc2f7f-8c45-4c2d-ba7b-2144fcbfd757
8e06675285464030a9fd7badf4b0939cba85e018ab8a20ea8cb911222fef7771
04e7fe0cfb19418a9ec2eacfe1d334d5 - - -] Releasing semaphore
"refresh_cache-9debce4f-ee92-4982-815f-769380728951" lock
/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:228
2016-07-05 15:40:32.702 99291 DEBUG oslo_concurrency.processutils
[req-41fc2f7f-8c45-4c2d-ba7b-2144fcbfd757
8e06675285464030a9fd7badf4b0939cba85e018ab8a20ea8cb911222fef7771
04e7fe0cfb19418a9ec2eacfe1d334d5 - - -] Running cmd (subprocess): rm -rf
/var/lib/nova/instances/9debce4f-ee92-4982-815f-769380728951_resize execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:326
2016-07-05 15:40:32.728 99291 DEBUG oslo_concurrency.processutils
[req-41fc2f7f-8c45-4c2d-ba7b-2144fcbfd757
8e06675285464030a9fd7badf4b0939cba85e018ab8a20ea8cb911222fef7771
04e7fe0cfb19418a9ec2eacfe1d334d5 - - -] CMD "rm -rf
/var/lib/nova/instances/9debce4f-ee92-4982-815f-769380728951_resize" returned:
0 in 0.026s execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:356
2016-07-05 15:40:32.741 99291 DEBUG nova.virt.libvirt.storage.rbd_utils
[req-41fc2f7f-8c45-4c2d-ba7b-2144fcbfd757
8e06675285464030a9fd7badf4b0939cba85e018ab8a20ea8cb911222fef7771
04e7fe0cfb19418a9ec2eacfe1d334d5 - - -] rbd image
9debce4f-ee92-4982-815f-769380728951_disk does not exist __init__
/usr/lib/python2.7/dist-packages/nova/virt/libvirt/storage/rbd_utils.py:64
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager
[req-41fc2f7f-8c45-4c2d-ba7b-2144fcbfd757
8e06675285464030a9fd7badf4b0939cba85e018ab8a20ea8cb911222fef7771
04e7fe0cfb19418a9ec2eacfe1d334d5 - - -] [instance:
9debce4f-ee92-4982-815f-769380728951] Setting instance vm_state to ERROR
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] Traceback (most recent call last):
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6588, in
_error_out_instance_on_exception
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] yield
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3531, in
_confirm_resize
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] network_info)
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 7484, in
confirm_migration
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] self._cleanup_resize(instance,
network_info)
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1258, in
_cleanup_resize
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] ignore_errors=True)
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 941,
in remove_snap
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] return
self.driver.remove_snap(self.rbd_name, name, ignore_errors)
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/storage/rbd_utils.py", line
396, in remove_snap
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] with RBDVolumeProxy(self,
str(volume), pool=pool) as vol:
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/storage/rbd_utils.py", line
65, in __init__
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] driver._disconnect_from_rados(client,
ioctx)
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] self.force_reraise()
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] six.reraise(self.type_, self.value,
self.tb)
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/storage/rbd_utils.py", line
61, in __init__
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] read_only=read_only)
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] File "rbd.pyx", line 1042, in
rbd.Image.__init__ (/tmp/buildd/ceph-10.2.2/src/build/rbd.c:9862)
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951] ImageNotFound: error opening image
9debce4f-ee92-4982-815f-769380728951_disk at snapshot None
2016-07-05 15:40:32.744 99291 ERROR nova.compute.manager [instance:
9debce4f-ee92-4982-815f-769380728951]
2016-07-05 15:40:32.811 99291 DEBUG oslo_concurrency.lockutils
[req-41fc2f7f-8c45-4c2d-ba7b-2144fcbfd757
8e06675285464030a9fd7badf4b0939cba85e018ab8a20ea8cb911222fef7771
04e7fe0cfb19418a9ec2eacfe1d334d5 - - -] Lock
"9debce4f-ee92-4982-815f-769380728951" released by
"nova.compute.manager.do_confirm_resize" :: held 0.507s inner
/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:285
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
[req-41fc2f7f-8c45-4c2d-ba7b-2144fcbfd757
8e06675285464030a9fd7badf4b0939cba85e018ab8a20ea8cb911222fef7771
04e7fe0cfb19418a9ec2eacfe1d334d5 - - -] Exception during message handling:
error opening image 9debce4f-ee92-4982-815f-769380728951_disk at snapshot None
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher Traceback
(most recent call last):
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 138,
in _dispatch_and_reply
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
incoming.message))
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 185,
in _dispatch
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher return
self._do_dispatch(endpoint, method, ctxt, args)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 127,
in _do_dispatch
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher result
= func(ctxt, **new_args)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/exception.py", line 110, in wrapped
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher payload)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
self.force_reraise()
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
six.reraise(self.type_, self.value, self.tb)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/exception.py", line 89, in wrapped
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher return
f(self, context, *args, **kw)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 409, in
decorated_function
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher return
function(self, context, *args, **kwargs)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 387, in
decorated_function
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
kwargs['instance'], e, sys.exc_info())
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
self.force_reraise()
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
six.reraise(self.type_, self.value, self.tb)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 375, in
decorated_function
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher return
function(self, context, *args, **kwargs)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3507, in
confirm_resize
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
do_confirm_resize(context, instance, migration.id)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 274, in
inner
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher return
f(*args, **kwargs)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3505, in
do_confirm_resize
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
migration=migration)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3531, in
_confirm_resize
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
network_info)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 7484, in
confirm_migration
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
self._cleanup_resize(instance, network_info)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1258, in
_cleanup_resize
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
ignore_errors=True)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 941,
in remove_snap
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher return
self.driver.remove_snap(self.rbd_name, name, ignore_errors)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/storage/rbd_utils.py", line
396, in remove_snap
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher with
RBDVolumeProxy(self, str(volume), pool=pool) as vol:
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/storage/rbd_utils.py", line
65, in __init__
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
driver._disconnect_from_rados(client, ioctx)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
self.force_reraise()
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
six.reraise(self.type_, self.value, self.tb)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/storage/rbd_utils.py", line
61, in __init__
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
read_only=read_only)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher File
"rbd.pyx", line 1042, in rbd.Image.__init__
(/tmp/buildd/ceph-10.2.2/src/build/rbd.c:9862)
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
ImageNotFound: error opening image 9debce4f-ee92-4982-815f-769380728951_disk at
snapshot None
2016-07-05 15:40:32.839 99291 ERROR oslo_messaging.rpc.dispatcher
openstack server show 9debce4f-ee92-4982-815f-769380728951
+--------------------------------------+------------------------------------------------------------------+
| Field | Value
|
+--------------------------------------+------------------------------------------------------------------+
| OS-DCF:diskConfig | AUTO
|
| OS-EXT-AZ:availability_zone | lab
|
| OS-EXT-SRV-ATTR:host | h12
|
| OS-EXT-SRV-ATTR:hypervisor_hostname | h12.example.com
|
| OS-EXT-SRV-ATTR:instance_name | instance-00000150
|
| OS-EXT-STS:power_state | 4
|
| OS-EXT-STS:task_state | None
|
| OS-EXT-STS:vm_state | stopped
|
| OS-SRV-USG:launched_at | 2016-07-05T06:40:11.000000
|
| OS-SRV-USG:terminated_at | None
|
| accessIPv4 |
|
| accessIPv6 |
|
| addresses | 1st-network=172.16.0.12
|
| config_drive |
|
| created | 2016-07-05T03:32:37Z
|
| flavor | m1.large (4)
|
| hostId |
90f85ee8c380c68c55b169ff1dc9917729638d0e744020641ace2700 |
| id | 9debce4f-ee92-4982-815f-769380728951
|
| image |
|
| key_name | desktop
|
| name | e1
|
| os-extended-volumes:volumes_attached | [{u'id':
u'90ea9f31-57b4-4162-8986-47065b23f407'}] |
| project_id | 04e7fe0cfb19418a9ec2eacfe1d334d5
|
| properties |
|
| security_groups | [{u'name': u'default'}]
|
| status | SHUTOFF
|
| updated | 2016-07-05T07:04:11Z
|
| user_id |
8e06675285464030a9fd7badf4b0939cba85e018ab8a20ea8cb911222fef7771 |
+--------------------------------------+------------------------------------------------------------------+
openstack volume show 90ea9f31-57b4-4162-8986-47065b23f407
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value
|
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attachments | [{u'server_id':
u'9debce4f-ee92-4982-815f-769380728951', u'attachment_id':
u'd119c454-65f4-4c7b-8135-3238a3e7a424', u'attached_at':
u'2016-07-05T03:33:37.127003', u'host_name': None, u'volume_id':
u'90ea9f31-57b4-4162-8986-47065b23f407', u'device': u'/dev/vda', u'id':
u'90ea9f31-57b4-4162-8986-47065b23f407'}]
|
| availability_zone | nova
|
| bootable | true
|
| consistencygroup_id | None
|
| created_at | 2016-07-05T03:32:38.353914
|
| description |
|
| encrypted | False
|
| id | 90ea9f31-57b4-4162-8986-47065b23f407
|
| migration_status | None
|
| multiattach | False
|
| name |
|
| os-vol-host-attr:host | mitaka-cinder@ceph#ceph
|
| os-vol-mig-status-attr:migstat | None
|
| os-vol-mig-status-attr:name_id | None
|
| os-vol-tenant-attr:tenant_id | 04e7fe0cfb19418a9ec2eacfe1d334d5
|
| properties | attached_mode='rw', readonly='False'
|
| replication_status | disabled
|
| size | 40
|
| snapshot_id | None
|
| source_volid | None
|
| status | in-use
|
| type | ceph
|
| updated_at | 2016-07-05T03:33:37.161179
|
| user_id |
8e06675285464030a9fd7badf4b0939cba85e018ab8a20ea8cb911222fef7771
|
| volume_image_metadata | {u'description': u'',
u'os_require_quiesce': u'yes', u'checksum':
u'cb3d84ccc990821e537fbe5227b3dbc3', u'hw_qemu_guest_agent': u'yes',
u'disk_format': u'raw', u'image_name': u'os2-2016062303', u'hw_scsi_model':
u'virtio-scsi', u'image_id': u'083aadf6-6a4c-45aa-aab6-66b6d2b2c2ba',
u'min_ram': u'0', u'container_format': u'bare', u'min_disk': u'3',
u'hw_disk_bus': u'scsi', u'size': u'2705326080'} |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
# On the Ceph cluster
⟫ sudo rbd ls openstack-volume | grep
9debce4f-ee92-4982-815f-769380728951_disk
1 ⟫ sudo rbd ls openstack-volume | grep 90ea9f31-57b4-4162-8986-47065b23f407
volume-90ea9f31-57b4-4162-8986-47065b23f407
⟫ sudo rbd info -p openstack-volume
volume-90ea9f31-57b4-4162-8986-47065b23f407
rbd image 'volume-90ea9f31-57b4-4162-8986-47065b23f407':
size 40960 MB in 10240 objects
order 22 (4096 kB objects)
block_name_prefix: rbd_data.b412b0238e1f29
format: 2
features: layering, exclusive-lock, object-map, fast-diff,
deep-flatten
flags:
(The volume size has not been changed.)
As you can see above, nova-compute tries to do something on the
9debce4f-ee92-4982-815f-769380728951_disk which doesn't actually
exist, recognizing it as an ephemeral disk.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1599067/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
