[CC to qemu-devel since I'm obviously doing something wrong here,
I'm just not sure what.]
I was getting ready to add multiple threads to ‘qemu-img convert’ (the
longest part of v2v conversions) when I noticed that it had them
already! (To be fair this was only added in February this year so no
wonder we didn't notice.)
To enable parallel convert we would need to use the ‘qemu-img convert
-W’ option (which allows out-of-order writes to the destination) and
‘qemu-img convert -m <#num-coroutines>’ option to select the
parallelism (defaults to 8). The documentation refers to coroutines
but I verified from strace that it is using real threads.
I did some testing to see what effect this has. For this I used a
large guest image which is approximately a third full of random data
(the rest being sparsely allocated):
Source format: raw
Source virtual size: 100 GB
Source used space: 31 GB
Target format: qcow2
Version: qemu-img-2.10.0-7.fc28.x86_64
Conversion command:
rm -f /to/target
time qemu-img convert [-W] [-m ##] -f raw source.img -O qcow2 /to/target
Source and target are regular files on two different disks. The test
machine is a Xeon E5 with 16 real cores.
----------------------------------------------------------------------
Non-preallocated output
(times are in seconds)
without -W -W
-m 1 153 -
-m 4 155 157
-m 8 [default] 158 231
-m 16 [max] 166 166
----------------------------------------------------------------------
The documentation for ‘-W’ notes that this is only recommended for
preallocated outputs (which the test above does not use). So let's
try using a preallocated qcow2 output.
Conversion command:
# the same target file is reused each time
time qemu-img convert -n [-W] [-m ##] -f raw source.img -O qcow2 /to/target
----------------------------------------------------------------------
Preallocated output
(times are in seconds)
without -W -W
-m 1 147 -
-m 4 146 145
-m 8 [default] 146 199
-m 16 [max] 147 146
----------------------------------------------------------------------
Based on this there seems to be some issue with the ‘-W’ option -- I
even thought I might have it backwards, but checking the code it does
seem like ‘-W’ enables (rather than disables) out of order writes.
Also some bizarre interaction between ‘-W’ and ‘-m 8’.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
