Re: [libvirt] Changing the default for qcow2 creation
On 10/25/2010 08:06 AM, Richard W.M. Jones wrote: Only step 4 is timed. I ran the program against test.img prepared in 4 different ways: 1. A sparse raw file:15.3 seconds truncate --size 300M test.img 2. A preallocated raw file: 14.8 seconds fallocate -l 300M test.img 3. A sparse qcow2 file: 223.0 seconds qemu-img create -f qcow2 test.img 300M 4. A metadata preallocated qcow2 file:14.5 seconds qemu-img create -f qcow2 -o preallocated=metadata test.img 300M libvirt's qemu driver doesn't currently preallocate qcow2 metadata when creating a new image. Given the tiny disk space overhead of the metadata (0.02%) and the small processing overhead of pre-creation relative to subsequent creation on-the-fly, I suggest that the libvirt qemu driver is updated to pre-allocate metadata by default. Thoughts? Your test might run faster if you did: $g-copy_size (/dev/zero, /test, size of file); However I think making the change is a no-brainer. We should add this flag by default. Is this something that we should just globally enable if qemu supports it, or do we need some XML tunable to allow someone to skip the pre-allocation for some reason? -- Eric Blake ebl...@redhat.com+1-801-349-2682 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] Changing the default for qcow2 creation
On Mon, Oct 25, 2010 at 02:40:41PM +0100, Matthew Booth wrote: I've recently been investigating a severe performance issue I noticed when writing to a qcow2-backed image. When virt-v2v is doing a format conversion from raw to qcow2, it does the following: 1. Create a new qcow2 image 2. Launch a libguestfs appliance (kvm) using the new image 3. Write the source raw data to the appliance's block device I noticed that the same process writing to a raw image rather than a qcow2 image was adequately fast, and decided to do some testing. I've attached my simple test program.[1] It does the following: 1. Start an appliance with test.img as a disk. 2. Format test.img with ext2. 3. Create a file /test 4. Write 256M of data to /test in 2M chunks Only step 4 is timed. I ran the program against test.img prepared in 4 different ways: 1. A sparse raw file:15.3 seconds truncate --size 300M test.img 2. A preallocated raw file: 14.8 seconds fallocate -l 300M test.img 3. A sparse qcow2 file: 223.0 seconds qemu-img create -f qcow2 test.img 300M 4. A metadata preallocated qcow2 file: 14.5 seconds qemu-img create -f qcow2 -o preallocated=metadata test.img 300M With the exception of (3), I ran the test 3 times and took the middle time rounded to 1DP. I saw about 5-10% variation. I only ran the test against (3) once. The precise ordering of 1, 2 and 4 is surprising, but given the variation probably not that interesting: they're all about the same. The interesting thing is that the overhead of qcow2 metadata creation during the test seems to account for a 15x performance penalty. I had a cursory look at metadata preallocation, which I hadn't been aware of before today. Creating a qcow2 image of any size with no preallocation results in a 136k file. If you preallocate the metadata, a sparse file is created large enough to accomodate the entire image, with 136k actually used. In the above 300M case this is 204k. On a slightly more practical 20G image, 3.3M is preallocated. It's also worth noting that the image takes considerably longer to create. On my laptop, creation without preallocation is 'instantaneous' at any size. With preallocation, a 20G image takes 6 seconds to create, and a 100G image takes 26 seconds. libvirt's qemu driver doesn't currently preallocate qcow2 metadata when creating a new image. Given the tiny disk space overhead of the metadata (0.02%) and the small processing overhead of pre-creation relative to subsequent creation on-the-fly, I suggest that the libvirt qemu driver is updated to pre-allocate metadata by default. Thoughts? Your test might run faster if you did: $g-copy_size (/dev/zero, /test, size of file); However I think making the change is a no-brainer. We should add this flag by default. [1] Note that I'm running this against libguestfs from git, which uses virtio-serial rather than usermode networking for appliance-host communication. This change alone improved the performance of this test by about 10x. If your numbers don't match mine, that's probably why. I don't know off the top of my head if this change has made it into F14 yet. It's definitely not in F13. F13 and F14 both use virtio-serial now. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] Changing the default for qcow2 creation
On 10/26/2010 01:06 AM, Richard W.M. Jones wrote: snip 4. A metadata preallocated qcow2 file:14.5 seconds qemu-img create -f qcow2 -o preallocated=metadata test.img 300M Just tested this on a F13 host, and it didn't like the the preallocated option: $ qemu-img create -f qcow2 -o preallocated=metadata testvm.qcow2 20G Unknown option 'preallocated' qemu-img: Invalid options for file format 'qcow2'. $ Looks like we'd need to add some kind of testing for the capability first. :) -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] Changing the default for qcow2 creation
On Tue, Oct 26, 2010 at 02:11:22AM +1100, Justin Clift wrote: On 10/26/2010 01:06 AM, Richard W.M. Jones wrote: snip 4. A metadata preallocated qcow2 file: 14.5 seconds qemu-img create -f qcow2 -o preallocated=metadata test.img 300M Just tested this on a F13 host, and it didn't like the the preallocated option: $ qemu-img create -f qcow2 -o preallocated=metadata testvm.qcow2 20G Unknown option 'preallocated' qemu-img: Invalid options for file format 'qcow2'. $ Looks like we'd need to add some kind of testing for the capability first. :) The option should be '-o preallocation=metadata'. This works for me on Fedora 13 too. This option was introduced to qemu upstream in commit a35e1c177debb01240243bd656caca302410d38c (Aug 17 2009). You could actually detect this at runtime by running: $ qemu-img create -f qcow2 -o preallocation=metadata /dev/null 1M Formatting '/dev/null', fmt=qcow2 size=1048576 encryption=off cluster_size=0 preallocation='metadata' $ echo $? 0 $ qemu-img create -f qcow2 -o preallocation=foobar /dev/null 1M Formatting '/dev/null', fmt=qcow2 size=1048576 encryption=off cluster_size=0 preallocation='foobar' Invalid preallocation mode: 'foobar' qemu-img: Error while formatting $ echo $? 1 However the first command segfaults on Fedora 14 (so near and yet so far!) I'm going to file a bug about that. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
