Public bug reported:
Environment:
Nova on Pike and Queens with kvm/libvirt compute driver
Versions tested:
ii nova-common 2:17.0.0-0ubuntu1~cloud0
all OpenStack Compute - common files
ii nova-compute 2:17.0.0-0ubuntu1~cloud0
all OpenStack Compute - compute node base
ii nova-compute-kvm 2:17.0.0-0ubuntu1~cloud0
all OpenStack Compute - compute node (KVM)
ii nova-compute-libvirt 2:17.0.0-0ubuntu1~cloud0
all OpenStack Compute - compute node libvirt support
ii python-nova 2:17.0.0-0ubuntu1~cloud0
all OpenStack Compute Python libraries
ii python-novaclient 2:9.1.1-0ubuntu1~cloud0
all client library for OpenStack Compute API - Python 2.7
ii python3-novaclient 2:9.1.1-0ubuntu1~cloud0
all client library for OpenStack Compute API - 3.x
Storage on cinder and lvm backend
ii cinder-common 2:12.0.0-0ubuntu1~cloud0
all Cinder storage service - common files
ii cinder-volume 2:12.0.0-0ubuntu1~cloud0
all Cinder storage service - Volume server
ii python-cinder 2:12.0.0-0ubuntu1~cloud0
all Cinder Python libraries
ii python-cinderclient 1:3.5.0-0ubuntu1~cloud0
all Python bindings to the OpenStack Volume API - Python 2.x
ii python3-cinderclient 1:3.5.0-0ubuntu1~cloud0
all Python bindings to the OpenStack Volume API - Python 3.x
Network with Neutron
ii neutron-common 2:12.0.0-0ubuntu1~cloud0
all Neutron is a virtual network service for Openstack - common
rc neutron-linuxbridge-agent 2:11.0.2-0ubuntu1.1~cloud0
all Neutron is a virtual network service for Openstack -
linuxbridge agent
ii neutron-openvswitch-agent 2:12.0.0-0ubuntu1~cloud0
all Neutron is a virtual network service for Openstack - Open
vSwitch plugin agent
ii python-neutron 2:12.0.0-0ubuntu1~cloud0
all Neutron is a virtual network service for Openstack - Python
library
ii python-neutron-fwaas 1:12.0.0-0ubuntu1~cloud0
all Firewall-as-a-Service driver for OpenStack Neutron
ii python-neutron-lib 1.13.0-0ubuntu1~cloud0
all Neutron shared routines and utilities - Python 2.7
ii python-neutronclient 1:6.7.0-0ubuntu1~cloud0
all client API library for Neutron - Python 2.7
ii python3-neutronclient 1:6.7.0-0ubuntu1~cloud0
all client API library for Neutron - Python 3.x
Steps to reproduce:
1.) Start a nova instance with a flavor which has a swap volume
Root Volume is created as /dev/cinder-volume/volume-UUID
The swap volume is created as /dev/cinder-volume/<UUID>_disk.swap
2.) Resize the instance to a different flavor with swap storage via
horizon or CLI
Expected Result:
1.) The swap volume on Node 1 is dropped
2.) A new swap volume is created on Node 2 and the instance is
migrated/restarted to node 2
Actual result:
1.) The swap volume is tried to be COPIED via scp from Node 1 to Node 2
2.) The source address is assumed to be /var/lib/nova/instances/<uuid>_disk.swap
3.) This fails with 'No such file or directory'(SIC) as swap volume is
/dev/cinder-volume/<UUID>_disk.swap
4.) Resizing leaves instance in error mode
Logfile:
(...)
2018-03-12 17:08:51.771 15427 DEBUG os_brick.initiator.connectors.iscsi
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] <== disconnect_volume:
return (678ms) None trace_logging_wrapper
/usr/lib/python2.7/dist-packages/os_brick/utils.py:170
2018-03-12 17:08:51.772 15427 DEBUG nova.virt.libvirt.volume.iscsi
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] Disconnected iSCSI Volume
disconnect_volume
/usr/lib/python2.7/dist-packages/nova/virt/libvirt/volume/iscsi.py:78
2018-03-12 17:08:51.774 15427 DEBUG nova.virt.libvirt.driver
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] skipping disk /dev/sdo
(vda) as it is a volume _get_instance_disk_info_from_config
/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py:7722
2018-03-12 17:08:51.774 15594 DEBUG oslo.privsep.daemon
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] privsep:
request[139660391538352]: (3, 'nova.privsep.fs.blockdev_size',
('/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap',), {})
loop /usr/lib/python2.7/dist-packages/oslo_privsep/daemon.py:443
2018-03-12 17:08:51.775 15594 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Running cmd (subprocess):
blockdev --getsize64
/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:372
2018-03-12 17:08:51.781 15594 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] CMD "blockdev --getsize64
/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap" returned: 0
in 0.007s execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:409
2018-03-12 17:08:51.782 15594 DEBUG oslo.privsep.daemon
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] privsep:
reply[139660391538352]: (4, ('2097152000\n', '')) loop
/usr/lib/python2.7/dist-packages/oslo_privsep/daemon.py:456
2018-03-12 17:08:51.783 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Running cmd (subprocess):
mv /var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:372
2018-03-12 17:08:51.798 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] CMD "mv
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize" returned:
0 in 0.015s execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:409
2018-03-12 17:08:51.799 15427 DEBUG nova.virt.libvirt.volume.remotefs
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Copying file
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
to
10.0.88.31:/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
copy_file
/usr/lib/python2.7/dist-packages/nova/virt/libvirt/volume/remotefs.py:107
2018-03-12 17:08:51.799 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Running cmd (subprocess):
scp -r
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
10.0.88.31:/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
execute /usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:372
2018-03-12 17:08:51.965 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] CMD "scp -r
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
10.0.88.31:/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap"
returned: 1 in 0.166s execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:409
2018-03-12 17:08:51.966 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] u'scp -r
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
10.0.88.31:/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap'
failed. Not Retrying. execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:457
2018-03-12 17:08:51.967 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Running cmd (subprocess):
rm -rf /var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82 execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:372
2018-03-12 17:08:51.983 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] CMD "rm -rf
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82" returned: 0 in
0.016s execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:409
2018-03-12 17:08:51.984 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Running cmd (subprocess):
mv /var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82 execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:372
2018-03-12 17:08:51.999 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] CMD "mv
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82" returned: 0 in
0.015s execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:409
2018-03-12 17:08:52.000 15427 DEBUG nova.virt.libvirt.volume.remotefs
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Removing directory
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82 on remote host
10.0.88.31 remove_dir
/usr/lib/python2.7/dist-packages/nova/virt/libvirt/volume/remotefs.py:101
2018-03-12 17:08:52.000 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Running cmd (subprocess):
ssh -o BatchMode=yes 10.0.88.31 rm -rf
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82 execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:372
2018-03-12 17:08:52.132 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] CMD "ssh -o BatchMode=yes
10.0.88.31 rm -rf /var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82"
returned: 0 in 0.132s execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:409
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] Setting instance vm_state to ERROR:
ProcessExecutionError: Unexpected error while running command.
Command: scp -r
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
10.0.88.31:/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
Exit code: 1
Stdout: u''
Stderr:
u'/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap:
No such file or directory\n'
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] Traceback (most recent call last):
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 7395, in
_error_out_instance_on_exception
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] yield
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4247, in
resize_instance
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] timeout, retry_interval)
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 8039, in
migrate_disk_and_power_off
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] shared_storage)
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] self.force_reraise()
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] six.reraise(self.type_, self.value,
self.tb)
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 8024, in
migrate_disk_and_power_off
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] compression=compression)
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/utils.py", line 226, in
copy_image
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] compression=compression)
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/volume/remotefs.py", line
110, in copy_file
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] compression=compression)
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/volume/remotefs.py", line
196, in copy_file
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] on_execute=on_execute,
on_completion=on_completion)
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/nova/utils.py", line 231, in execute
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] return processutils.execute(*cmd,
**kwargs)
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py", line 424,
in execute
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] cmd=sanitized_cmd)
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] ProcessExecutionError: Unexpected error
while running command.
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] Command: scp -r
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
10.0.88.31:/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] Exit code: 1
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] Stdout: u''
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] Stderr:
u'/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap:
No such file or directory\n'
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82]
2018-03-12 17:08:52.696 15427 DEBUG oslo_concurrency.lockutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Lock "compute_resources"
acquired by "nova.compute.resource_tracker.update_usage" :: waited 0.000s inner
/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:276
2018-03-12 17:08:52.758 15594 DEBUG oslo.privsep.daemon
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] privsep:
request[139660391538352]: (3, 'nova.privsep.fs.vginfo', ('cinder-volumes',),
{}) loop /usr/lib/python2.7/dist-packages/oslo_privsep/daemon.py:443
2018-03-12 17:08:52.758 15594 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Running cmd (subprocess):
vgs --noheadings --nosuffix --separator | --units b -o vg_size,vg_free
cinder-volumes execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:372
2018-03-12 17:08:52.783 15594 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] CMD "vgs --noheadings
--nosuffix --separator | --units b -o vg_size,vg_free cinder-volumes" returned:
0 in 0.024s execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:409
2018-03-12 17:08:52.783 15594 DEBUG oslo.privsep.daemon
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] privsep:
reply[139660391538352]: (4, (' 3000454086656|2075542945792\n', '')) loop
/usr/lib/python2.7/dist-packages/oslo_privsep/daemon.py:456
2018-03-12 17:08:52.799 15427 DEBUG nova.compute.provider_tree
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Updating resource provider
1c1ad99d-3976-4aa8-bf32-3394f6c2040f generation from 53 to 54
_update_generation
/usr/lib/python2.7/dist-packages/nova/compute/provider_tree.py:153
2018-03-12 17:08:52.800 15427 DEBUG oslo_concurrency.lockutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Lock "compute_resources"
released by "nova.compute.resource_tracker.update_usage" :: held 0.104s inner
/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:288
2018-03-12 17:08:52.800 15427 INFO nova.compute.manager
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] Successfully reverted task state from
resize_migrating on failure for instance.
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Exception during message
handling: ProcessExecutionError: Unexpected error while running command.
Command: scp -r
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
10.0.88.31:/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
Exit code: 1
Stdout: u''
Stderr:
u'/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap:
No such file or directory\n'
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server Traceback (most
recent call last):
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in
_process_incoming
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server res =
self.dispatcher.dispatch(message)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220,
in dispatch
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server return
self._do_dispatch(endpoint, method, ctxt, args)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190,
in _do_dispatch
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server result =
func(ctxt, **new_args)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 76, in
wrapped
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
function_name, call_dict, binary)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
self.force_reraise()
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
six.reraise(self.type_, self.value, self.tb)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 67, in
wrapped
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server return
f(self, context, *args, **kw)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 186, in
decorated_function
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server "Error: %s",
e, instance=instance)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
self.force_reraise()
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
six.reraise(self.type_, self.value, self.tb)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 156, in
decorated_function
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server return
function(self, context, *args, **kwargs)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/compute/utils.py", line 976, in
decorated_function
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server return
function(self, context, *args, **kwargs)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 214, in
decorated_function
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
kwargs['instance'], e, sys.exc_info())
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
self.force_reraise()
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
six.reraise(self.type_, self.value, self.tb)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 202, in
decorated_function
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server return
function(self, context, *args, **kwargs)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4247, in
resize_instance
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server timeout,
retry_interval)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 8039, in
migrate_disk_and_power_off
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
shared_storage)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
self.force_reraise()
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
six.reraise(self.type_, self.value, self.tb)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 8024, in
migrate_disk_and_power_off
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
compression=compression)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/utils.py", line 226, in
copy_image
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
compression=compression)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/volume/remotefs.py", line
110, in copy_file
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
compression=compression)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/volume/remotefs.py", line
196, in copy_file
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
on_execute=on_execute, on_completion=on_completion)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/utils.py", line 231, in execute
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server return
processutils.execute(*cmd, **kwargs)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py", line 424,
in execute
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
cmd=sanitized_cmd)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
ProcessExecutionError: Unexpected error while running command.
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server Command: scp -r
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
10.0.88.31:/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server Exit code: 1
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server Stdout: u''
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server Stderr:
u'/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap:
No such file or directory\n'
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
2018-03-12 17:09:06.091 15427 DEBUG nova.virt.driver
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Emitting event
<LifecycleEvent: 1520870931.09, 1ae5b6ee-aa77-4661-8542-58bd9da5ef82 =>
Stopped> emit_event /usr/lib/python2.7/dist-packages/nova/virt/driver.py:1514
2018-03-12 17:09:06.092 15427 INFO nova.compute.manager
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] VM Stopped (Lifecycle Event)
Instance need then to be destroyed and removed
Br br
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1755266
Title:
Instance resize with swap on cinder volume fails
Status in OpenStack Compute (nova):
New
Bug description:
Environment:
Nova on Pike and Queens with kvm/libvirt compute driver
Versions tested:
ii nova-common 2:17.0.0-0ubuntu1~cloud0
all OpenStack Compute - common files
ii nova-compute 2:17.0.0-0ubuntu1~cloud0
all OpenStack Compute - compute node base
ii nova-compute-kvm 2:17.0.0-0ubuntu1~cloud0
all OpenStack Compute - compute node (KVM)
ii nova-compute-libvirt 2:17.0.0-0ubuntu1~cloud0
all OpenStack Compute - compute node libvirt support
ii python-nova 2:17.0.0-0ubuntu1~cloud0
all OpenStack Compute Python libraries
ii python-novaclient 2:9.1.1-0ubuntu1~cloud0
all client library for OpenStack Compute API - Python 2.7
ii python3-novaclient 2:9.1.1-0ubuntu1~cloud0
all client library for OpenStack Compute API - 3.x
Storage on cinder and lvm backend
ii cinder-common 2:12.0.0-0ubuntu1~cloud0
all Cinder storage service - common files
ii cinder-volume 2:12.0.0-0ubuntu1~cloud0
all Cinder storage service - Volume server
ii python-cinder 2:12.0.0-0ubuntu1~cloud0
all Cinder Python libraries
ii python-cinderclient 1:3.5.0-0ubuntu1~cloud0
all Python bindings to the OpenStack Volume API - Python 2.x
ii python3-cinderclient 1:3.5.0-0ubuntu1~cloud0
all Python bindings to the OpenStack Volume API - Python 3.x
Network with Neutron
ii neutron-common 2:12.0.0-0ubuntu1~cloud0
all Neutron is a virtual network service for Openstack - common
rc neutron-linuxbridge-agent 2:11.0.2-0ubuntu1.1~cloud0
all Neutron is a virtual network service for Openstack -
linuxbridge agent
ii neutron-openvswitch-agent 2:12.0.0-0ubuntu1~cloud0
all Neutron is a virtual network service for Openstack - Open
vSwitch plugin agent
ii python-neutron 2:12.0.0-0ubuntu1~cloud0
all Neutron is a virtual network service for Openstack - Python
library
ii python-neutron-fwaas 1:12.0.0-0ubuntu1~cloud0
all Firewall-as-a-Service driver for OpenStack Neutron
ii python-neutron-lib 1.13.0-0ubuntu1~cloud0
all Neutron shared routines and utilities - Python 2.7
ii python-neutronclient 1:6.7.0-0ubuntu1~cloud0
all client API library for Neutron - Python 2.7
ii python3-neutronclient 1:6.7.0-0ubuntu1~cloud0
all client API library for Neutron - Python 3.x
Steps to reproduce:
1.) Start a nova instance with a flavor which has a swap volume
Root Volume is created as /dev/cinder-volume/volume-UUID
The swap volume is created as /dev/cinder-volume/<UUID>_disk.swap
2.) Resize the instance to a different flavor with swap storage via
horizon or CLI
Expected Result:
1.) The swap volume on Node 1 is dropped
2.) A new swap volume is created on Node 2 and the instance is
migrated/restarted to node 2
Actual result:
1.) The swap volume is tried to be COPIED via scp from Node 1 to Node 2
2.) The source address is assumed to be
/var/lib/nova/instances/<uuid>_disk.swap
3.) This fails with 'No such file or directory'(SIC) as swap volume is
/dev/cinder-volume/<UUID>_disk.swap
4.) Resizing leaves instance in error mode
Logfile:
(...)
2018-03-12 17:08:51.771 15427 DEBUG os_brick.initiator.connectors.iscsi
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] <== disconnect_volume:
return (678ms) None trace_logging_wrapper
/usr/lib/python2.7/dist-packages/os_brick/utils.py:170
2018-03-12 17:08:51.772 15427 DEBUG nova.virt.libvirt.volume.iscsi
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] Disconnected iSCSI Volume
disconnect_volume
/usr/lib/python2.7/dist-packages/nova/virt/libvirt/volume/iscsi.py:78
2018-03-12 17:08:51.774 15427 DEBUG nova.virt.libvirt.driver
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] skipping disk /dev/sdo
(vda) as it is a volume _get_instance_disk_info_from_config
/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py:7722
2018-03-12 17:08:51.774 15594 DEBUG oslo.privsep.daemon
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] privsep:
request[139660391538352]: (3, 'nova.privsep.fs.blockdev_size',
('/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap',), {})
loop /usr/lib/python2.7/dist-packages/oslo_privsep/daemon.py:443
2018-03-12 17:08:51.775 15594 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Running cmd (subprocess):
blockdev --getsize64
/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:372
2018-03-12 17:08:51.781 15594 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] CMD "blockdev --getsize64
/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap" returned: 0
in 0.007s execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:409
2018-03-12 17:08:51.782 15594 DEBUG oslo.privsep.daemon
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] privsep:
reply[139660391538352]: (4, ('2097152000\n', '')) loop
/usr/lib/python2.7/dist-packages/oslo_privsep/daemon.py:456
2018-03-12 17:08:51.783 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Running cmd (subprocess):
mv /var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:372
2018-03-12 17:08:51.798 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] CMD "mv
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize" returned:
0 in 0.015s execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:409
2018-03-12 17:08:51.799 15427 DEBUG nova.virt.libvirt.volume.remotefs
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Copying file
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
to
10.0.88.31:/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
copy_file
/usr/lib/python2.7/dist-packages/nova/virt/libvirt/volume/remotefs.py:107
2018-03-12 17:08:51.799 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Running cmd (subprocess):
scp -r
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
10.0.88.31:/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
execute /usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:372
2018-03-12 17:08:51.965 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] CMD "scp -r
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
10.0.88.31:/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap"
returned: 1 in 0.166s execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:409
2018-03-12 17:08:51.966 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] u'scp -r
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
10.0.88.31:/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap'
failed. Not Retrying. execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:457
2018-03-12 17:08:51.967 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Running cmd (subprocess):
rm -rf /var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82 execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:372
2018-03-12 17:08:51.983 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] CMD "rm -rf
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82" returned: 0 in
0.016s execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:409
2018-03-12 17:08:51.984 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Running cmd (subprocess):
mv /var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82 execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:372
2018-03-12 17:08:51.999 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] CMD "mv
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82" returned: 0 in
0.015s execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:409
2018-03-12 17:08:52.000 15427 DEBUG nova.virt.libvirt.volume.remotefs
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Removing directory
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82 on remote host
10.0.88.31 remove_dir
/usr/lib/python2.7/dist-packages/nova/virt/libvirt/volume/remotefs.py:101
2018-03-12 17:08:52.000 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Running cmd (subprocess):
ssh -o BatchMode=yes 10.0.88.31 rm -rf
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82 execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:372
2018-03-12 17:08:52.132 15427 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] CMD "ssh -o BatchMode=yes
10.0.88.31 rm -rf /var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82"
returned: 0 in 0.132s execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:409
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] Setting instance vm_state to ERROR:
ProcessExecutionError: Unexpected error while running command.
Command: scp -r
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
10.0.88.31:/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
Exit code: 1
Stdout: u''
Stderr:
u'/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap:
No such file or directory\n'
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] Traceback (most recent call last):
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 7395, in
_error_out_instance_on_exception
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] yield
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4247, in
resize_instance
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] timeout, retry_interval)
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 8039, in
migrate_disk_and_power_off
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] shared_storage)
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] self.force_reraise()
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] six.reraise(self.type_, self.value,
self.tb)
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 8024, in
migrate_disk_and_power_off
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] compression=compression)
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/utils.py", line 226, in
copy_image
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] compression=compression)
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/volume/remotefs.py", line
110, in copy_file
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] compression=compression)
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/volume/remotefs.py", line
196, in copy_file
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] on_execute=on_execute,
on_completion=on_completion)
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/nova/utils.py", line 231, in execute
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] return processutils.execute(*cmd,
**kwargs)
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] File
"/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py", line 424,
in execute
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] cmd=sanitized_cmd)
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] ProcessExecutionError: Unexpected error
while running command.
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] Command: scp -r
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
10.0.88.31:/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] Exit code: 1
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] Stdout: u''
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] Stderr:
u'/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap:
No such file or directory\n'
2018-03-12 17:08:52.191 15427 ERROR nova.compute.manager [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82]
2018-03-12 17:08:52.696 15427 DEBUG oslo_concurrency.lockutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Lock "compute_resources"
acquired by "nova.compute.resource_tracker.update_usage" :: waited 0.000s inner
/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:276
2018-03-12 17:08:52.758 15594 DEBUG oslo.privsep.daemon
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] privsep:
request[139660391538352]: (3, 'nova.privsep.fs.vginfo', ('cinder-volumes',),
{}) loop /usr/lib/python2.7/dist-packages/oslo_privsep/daemon.py:443
2018-03-12 17:08:52.758 15594 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Running cmd (subprocess):
vgs --noheadings --nosuffix --separator | --units b -o vg_size,vg_free
cinder-volumes execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:372
2018-03-12 17:08:52.783 15594 DEBUG oslo_concurrency.processutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] CMD "vgs --noheadings
--nosuffix --separator | --units b -o vg_size,vg_free cinder-volumes" returned:
0 in 0.024s execute
/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:409
2018-03-12 17:08:52.783 15594 DEBUG oslo.privsep.daemon
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] privsep:
reply[139660391538352]: (4, (' 3000454086656|2075542945792\n', '')) loop
/usr/lib/python2.7/dist-packages/oslo_privsep/daemon.py:456
2018-03-12 17:08:52.799 15427 DEBUG nova.compute.provider_tree
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Updating resource provider
1c1ad99d-3976-4aa8-bf32-3394f6c2040f generation from 53 to 54
_update_generation
/usr/lib/python2.7/dist-packages/nova/compute/provider_tree.py:153
2018-03-12 17:08:52.800 15427 DEBUG oslo_concurrency.lockutils
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Lock "compute_resources"
released by "nova.compute.resource_tracker.update_usage" :: held 0.104s inner
/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:288
2018-03-12 17:08:52.800 15427 INFO nova.compute.manager
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] Successfully reverted task state from
resize_migrating on failure for instance.
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Exception during message
handling: ProcessExecutionError: Unexpected error while running command.
Command: scp -r
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
10.0.88.31:/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
Exit code: 1
Stdout: u''
Stderr:
u'/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap:
No such file or directory\n'
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server Traceback (most
recent call last):
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 163, in
_process_incoming
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server res =
self.dispatcher.dispatch(message)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 220,
in dispatch
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server return
self._do_dispatch(endpoint, method, ctxt, args)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 190,
in _do_dispatch
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server result =
func(ctxt, **new_args)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 76, in
wrapped
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
function_name, call_dict, binary)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
self.force_reraise()
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
six.reraise(self.type_, self.value, self.tb)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 67, in
wrapped
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server return
f(self, context, *args, **kw)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 186, in
decorated_function
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server "Error:
%s", e, instance=instance)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
self.force_reraise()
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
six.reraise(self.type_, self.value, self.tb)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 156, in
decorated_function
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server return
function(self, context, *args, **kwargs)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/compute/utils.py", line 976, in
decorated_function
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server return
function(self, context, *args, **kwargs)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 214, in
decorated_function
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
kwargs['instance'], e, sys.exc_info())
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
self.force_reraise()
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
six.reraise(self.type_, self.value, self.tb)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 202, in
decorated_function
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server return
function(self, context, *args, **kwargs)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4247, in
resize_instance
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server timeout,
retry_interval)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 8039, in
migrate_disk_and_power_off
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
shared_storage)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
self.force_reraise()
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
six.reraise(self.type_, self.value, self.tb)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 8024, in
migrate_disk_and_power_off
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
compression=compression)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/utils.py", line 226, in
copy_image
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
compression=compression)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/volume/remotefs.py", line
110, in copy_file
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
compression=compression)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/volume/remotefs.py", line
196, in copy_file
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
on_execute=on_execute, on_completion=on_completion)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/nova/utils.py", line 231, in execute
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server return
processutils.execute(*cmd, **kwargs)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server File
"/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py", line 424,
in execute
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
cmd=sanitized_cmd)
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
ProcessExecutionError: Unexpected error while running command.
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server Command: scp -r
/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
10.0.88.31:/dev/cinder-volumes/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server Exit code: 1
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server Stdout: u''
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server Stderr:
u'/var/lib/nova/instances/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_resize/1ae5b6ee-aa77-4661-8542-58bd9da5ef82_disk.swap:
No such file or directory\n'
2018-03-12 17:08:52.829 15427 ERROR oslo_messaging.rpc.server
2018-03-12 17:09:06.091 15427 DEBUG nova.virt.driver
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] Emitting event
<LifecycleEvent: 1520870931.09, 1ae5b6ee-aa77-4661-8542-58bd9da5ef82 =>
Stopped> emit_event /usr/lib/python2.7/dist-packages/nova/virt/driver.py:1514
2018-03-12 17:09:06.092 15427 INFO nova.compute.manager
[req-c85ef642-2708-4395-bcf4-3a3f8820a39e 9fce36209f42437bb9d4e5d4423586ae
87cbad1ec81143b5bbc557a40d81c81a - default default] [instance:
1ae5b6ee-aa77-4661-8542-58bd9da5ef82] VM Stopped (Lifecycle Event)
Instance need then to be destroyed and removed
Br br
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1755266/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
