Re: [Qemu-devel] [PATCH v1 00/13] qcow2: space preallocation and COW improvements
On 05/23/2017 05:35 PM, Eric Blake wrote: > On 05/19/2017 04:34 AM, Anton Nefedov wrote: >> This pull request is to address a few performance problems of qcow2 format: >> >> 1. non cluster-aligned write requests (to unallocated clusters) explicitly >> pad data with zeroes if there is no backing data. This can be avoided >> and the whole clusters are preallocated and zeroed in a single >> efficient write_zeroes() operation, also providing better host file >> continuity >> >> 2. moreover, efficient write_zeroes() operation can be used to preallocate >> space megabytes ahead which gives noticeable improvement on some storage >> types (e.g. distributed storages where space allocation operation is >> expensive) >> >> 3. preallocating/zeroing the clusters in advance makes possible to enable >> simultaneous writes to the same unallocated cluster, which is beneficial >> for parallel sequential write operations which are not cluster-aligned >> >> Performance test results are added to commit messages (see patch 3, 12) > And now Berto has posted parallel patches. I'm not sure which ones to > focus on, or if you can work it out between you on the best approach > forward... > > https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg05236.html > I have seen those patches and will comment. yes, they are intersect. He is concentrating on the fact of performing operation in minimal read/write amount. We have made an attempt to minimize amount of IO in the expand case. Frankly speaking Berto patches are not that influenced. We definitely should merge IO if write_zeroes are impossible. Thus both could be merged in parallel. What I want to note is the fact that there is another big problem in QCOW2 layer. We have 64k block by default. Guest request can be 4-5Mb in total. If the image is fragmented, most like it will if it is old, we will have sequentially read/write the data. Thus in ideal world we should switch to co-routine pool and the same approach would be the best in COW too. Pls note, that this also would very positively influence VM downtime on migration as bdrv_drain_all is one the most important time hogs. Den
Re: [Qemu-devel] [PATCH v1 00/13] qcow2: space preallocation and COW improvements
On 05/19/2017 04:34 AM, Anton Nefedov wrote: > This pull request is to address a few performance problems of qcow2 format: > > 1. non cluster-aligned write requests (to unallocated clusters) explicitly > pad data with zeroes if there is no backing data. This can be avoided > and the whole clusters are preallocated and zeroed in a single > efficient write_zeroes() operation, also providing better host file > continuity > > 2. moreover, efficient write_zeroes() operation can be used to preallocate > space megabytes ahead which gives noticeable improvement on some storage > types (e.g. distributed storages where space allocation operation is > expensive) > > 3. preallocating/zeroing the clusters in advance makes possible to enable > simultaneous writes to the same unallocated cluster, which is beneficial > for parallel sequential write operations which are not cluster-aligned > > Performance test results are added to commit messages (see patch 3, 12) And now Berto has posted parallel patches. I'm not sure which ones to focus on, or if you can work it out between you on the best approach forward... https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg05236.html -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org signature.asc Description: OpenPGP digital signature
[Qemu-devel] [PATCH v1 00/13] qcow2: space preallocation and COW improvements
This pull request is to address a few performance problems of qcow2 format: 1. non cluster-aligned write requests (to unallocated clusters) explicitly pad data with zeroes if there is no backing data. This can be avoided and the whole clusters are preallocated and zeroed in a single efficient write_zeroes() operation, also providing better host file continuity 2. moreover, efficient write_zeroes() operation can be used to preallocate space megabytes ahead which gives noticeable improvement on some storage types (e.g. distributed storages where space allocation operation is expensive) 3. preallocating/zeroing the clusters in advance makes possible to enable simultaneous writes to the same unallocated cluster, which is beneficial for parallel sequential write operations which are not cluster-aligned Performance test results are added to commit messages (see patch 3, 12) Anton Nefedov (9): qcow2: is_zero_sectors(): return true if area is outside of backing file qcow2: do not COW the empty areas qcow2: set inactive flag qcow2: handle_prealloc(): find out if area zeroed by earlier preallocation qcow2: fix misleading comment about L2 linking qcow2-cluster: slightly refactor handle_dependencies() qcow2-cluster: make handle_dependencies() logic easier to follow qcow2: allow concurrent unaligned writes to the same clusters iotest 046: test simultaneous cluster write error case Denis V. Lunev (3): qcow2: alloc space for COW in one chunk qcow2: preallocation at image expand qcow2: truncate preallocated space Pavel Butsykin (1): qcow2: check space leak at the end of the image block/qcow2-cache.c| 3 + block/qcow2-cluster.c | 216 +++- block/qcow2-refcount.c | 21 +++ block/qcow2.c | 286 - block/qcow2.h | 26 tests/qemu-iotests/026.out | 104 ++ tests/qemu-iotests/026.out.nocache | 104 ++ tests/qemu-iotests/029.out | 5 +- tests/qemu-iotests/046 | 38 - tests/qemu-iotests/046.out | 23 +++ tests/qemu-iotests/060 | 2 +- tests/qemu-iotests/060.out | 13 +- tests/qemu-iotests/061.out | 5 +- tests/qemu-iotests/066 | 2 +- tests/qemu-iotests/066.out | 9 +- tests/qemu-iotests/098.out | 7 +- tests/qemu-iotests/108.out | 5 +- tests/qemu-iotests/112.out | 5 +- tests/qemu-iotests/154.out | 4 +- 19 files changed, 769 insertions(+), 109 deletions(-) -- 2.7.4
