On Tue, Jun 16, 2015 at 01:42:02AM +0300, Pavel Boldin wrote:
[. . .]
> Michal Privoznik (3):
> virDomainDiskGetSource: Mark passed disk as 'const'
> qemuMigrationBeginPhase: Fix function header indentation
> qemuMigrationDriveMirror: Force raw format for NBD
>
> Pavel Boldin (6):
> util: multi-value virTypedParameter
> util: multi-value parameters in virTypedParamsAdd*
> util: virTypedParams{Filter,GetAllStrings}
> util: add virTypedParamsAddStringList
> qemu: migration: selective block device migration
> virsh: selective block device migration
>
> include/libvirt/libvirt-domain.h | 9 ++
> include/libvirt/libvirt-host.h | 11 ++
> src/conf/domain_conf.c | 2 +-
> src/conf/domain_conf.h | 2 +-
> src/libvirt_public.syms | 6 +
> src/qemu/qemu_driver.c | 78 ++++++++---
> src/qemu/qemu_migration.c | 264 +++++++++++++++++++++++++----------
> src/qemu/qemu_migration.h | 24 ++--
> src/util/virtypedparam.c | 259 +++++++++++++++++++++++++++-------
> src/util/virtypedparam.h | 19 +++
> tests/Makefile.am | 6 +
> tests/virtypedparamtest.c | 295
> +++++++++++++++++++++++++++++++++++++++
> tools/virsh-domain.c | 23 +++
> tools/virsh.pod | 21 +--
> 14 files changed, 854 insertions(+), 165 deletions(-)
> create mode 100644 tests/virtypedparamtest.c
>
New test with this revision of patches applied.
Test env
~~~~~~~~
- On source and destination host, libvirt is compiled with the above
patches:
$ git describe
v1.2.16-204-g7aee251
- Create SSH keys and copy to dest host:
# Create the SSH keys with empty passphrase
$ ssh-keygen -t rsa
# Copy the key to the remote host
$ ssh-copy-id root@devstack3
# `ssh root@devstack3` succeeds w/o password prompt
- Since I'm on a trusted network, on dest host:
$ cat /etc/libvirt/libvirtd.conf | grep -v ^$ | grep -v ^#
listen_tls = 0
listen_tcp = 1
auth_tcp = "none"
- Run the libvirtd daemon on destination (with "--listen" mode), as
root:
$ ./run daemon/libvirtd --listen &
Test migration
~~~~~~~~~~~~~~
On source (from newly built libvirtd), as root:
$ ./run tools/virsh list
I have two disks:
$ ./run tools/virsh domblklist cvm1
Target Source
------------------------------------------------
vda /var/lib/libvirt/images/cirros-0.3.3-x86_64-disk.img
vdb /export/disk2.img
So, let's try to migrate the 'vdb' disk:
$ ./virsh migrate --verbose --p2p --migratedisks vdb \
--live cvm1 qemu+ssh://root@devstack3/system
error: Timed out during operation: cannot acquire state change lock (held
by remoteDispatchDomainMigratePerform3Params)
>From libvirt debug logs
~~~~~~~~~~~~~~~~~~~~~~~
libvirtd debug log[1] from source (destination log is empty)):
[. . .]
2015-06-17 15:13:53.317+0000: 781: debug : virDomainMigratePerform3Params:5202
: dom=0x7f2118f13c40, (VM: name=cvm1,
uuid=ab4c412b-6fdc-4fc4-b78c-f1d49db10d4e),
dconnuri=qemu+tcp://root@devstack3/system, params=0x7f2118f12a90, nparams=1,
cookiein=(nil), cookieinlen=0, cookieout=0x7f2106f38ba8,
cookieoutlen=0x7f2106f38ba4, flags=3
2015-06-17 15:13:53.317+0000: 781: debug : virDomainMigratePerform3Params:5203
: params["migrate_disks"]=(string)vdb
2015-06-17 15:13:53.317+0000: 781: debug : qemuMigrationPerform:5238 :
driver=0x7f20f416b840, conn=0x7f20dc005c30, vm=0x7f20f41e9640, xmlin=<null>,
dconnuri=qemu+tcp://root@devstack3/system, uri=<null>, graphicsuri=<null>,
listenAddress=<null>, nmigrate_disks=1, migrate_disks=0x7f2118f13930,
cookiein=<null>, cookieinlen=0, cookieout=0x7f2106f38ba8,
cookieoutlen=0x7f2106f38ba4, flags=3, dname=<null>, resource=0, v3proto=1
2015-06-17 15:13:53.317+0000: 781: debug : qemuDomainObjBeginJobInternal:1397 :
Starting async job: none (async=migration out vm=0x7f20f41e9640 name=cvm1)
2015-06-17 15:13:53.317+0000: 781: debug : qemuDomainObjBeginJobInternal:1414 :
Waiting for async job (vm=0x7f20f41e9640 name=cvm1)
2015-06-17 15:13:53.821+0000: 782: debug : virThreadJobSet:96 : Thread 782
(virNetServerHandleJob) is now running job remoteDispatchDomainGetJobInfo
2015-06-17 15:13:53.821+0000: 782: debug : virDomainGetJobInfo:8808 :
dom=0x7f20dc008c30, (VM: name=cvm1, uuid=ab4c412b-6fdc-4fc4-b78c-f1d49db10d4e),
info=0x7f2106737b50
2015-06-17 15:13:53.821+0000: 782: debug : virThreadJobClear:121 : Thread 782
(virNetServerHandleJob) finished job remoteDispatchDomainGetJobInfo with ret=0
2015-06-17 15:13:54.325+0000: 780: debug : virThreadJobSet:96 : Thread 780
(virNetServerHandleJob) is now running job remoteDispatchDomainGetJobInfo
2015-06-17 15:13:54.325+0000: 780: debug : virDomainGetJobInfo:8808 :
dom=0x7f20dc008c30, (VM: name=cvm1, uuid=ab4c412b-6fdc-4fc4-b78c-f1d49db10d4e),
info=0x7f2107739b50
2015-06-17 15:13:54.325+0000: 780: debug : virThreadJobClear:121 : Thread 780
(virNetServerHandleJob) finished job remoteDispatchDomainGetJobInfo with ret=0
[. . .]
remoteDispatchDomainMigratePerform3Params, 784
remoteDispatchDomainMigratePerform3Params) for (520s, 520s)
2015-06-17 15:14:23.320+0000: 781: error : qemuDomainObjBeginJobInternal:1492 :
Timed out during operation: cannot acquire state change lock (held by
remoteDispatchDomainMigratePerform3Params)
2015-06-17 15:14:23.320+0000: 781: debug : virThreadJobClear:121 : Thread 781
(virNetServerHandleJob) finished job remoteDispatchDomainMigratePerform3Params
with ret=-1
2015-06-17 15:14:23.320+0000: 783: debug : virThreadJobSet:96 : Thread 783
(virNetServerHandleJob) is now running job remoteDispatchConnectClose
2015-06-17 15:14:23.320+0000: 783: debug : virThreadJobClear:121 : Thread 783
(virNetServerHandleJob) finished job remoteDispatchConnectClose with ret=0
How can I mitigate this? (I realize this is not due to these patches,
proably something with my test environment.)
Since this is non-shared storage migration, I tried to supply
'--copy-storage-inc' to no avail (same error as above).
Probably I should test by building local RPMs.
[1]
https://kashyapc.fedorapeople.org/virt/temp/libvirtd-log-selective-blockdev-failed.log
--
/kashyap
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
