On 06/26/2017 01:04 PM, Daniel P. Berrange wrote: > On Mon, Jun 26, 2017 at 11:20:33AM +0200, Peter Lieven wrote: >> Am 26.06.2017 um 10:28 schrieb Kevin Wolf: >>> [ Cc: qemu-devel; don't post to qemu-block only! ] >>> >>> Am 26.06.2017 um 09:57 hat Peter Lieven geschrieben: >>>> Hi, >>>> >>>> I am currently working on optimizing speed for compressed QCOW2 >>>> images. We use them for templates and would also like to use them for >>>> backups, but the latter is almost infeasible because using gzip for >>>> compression is horribly slow. I tried to experiment with different >>>> options to deflate, but in the end I think its better to use a >>>> different compression algorithm for cases where speed matters. As we >>>> already have probing for it in configure and as it is widely used I >>>> would like to use LZO for that purpose. I think it would be best to >>>> have a flag to indicate that compressed blocks use LZO compression, >>>> but I would need a little explaination which of the feature fields I >>>> have to use to prevent an older (incompatible) Qemu opening LZO >>>> compressed QCOW2 images. >>>> >>>> I also have already some numbers. I converted a fresh Debian 9 Install >>>> which has an uncomressed QCOW2 size of 1158 MB with qemu-img to a >>>> compressed QCOW2. With GZIP compression the result is 356MB whereas >>>> the LZO version is 452MB. However, the current GZIP variant uses 35 >>>> seconds for this operation where LZO only needs 4 seconds. I think is >>>> is a good trade in especially when its optional so the user can >>>> choose. >>>> >>>> What are your thoughts? >>> We had a related RFC patch by Den earlier this year, which never >>> received many comment and never got out of RFC: >>> >>> https://lists.gnu.org/archive/html/qemu-devel/2017-03/msg04682.html >> I was not aware of that one. Thanks for pointing out. >> >>> So he chose a different algorithm (zstd). When I asked, he posted a >>> comparison of algorithms (however a generic one and not measured in the >>> context of qemu) that suggests that LZO would be slightly faster, but >>> have a considerable worse compression ratio with the settings that were >>> benchmarked. >> My idea to choose LZO was that it is widely available and available in >> any distro you can think of. We already have probing for it in configure. >> My concern with ZSTD would be that it seems there are no packages >> available for most distros and that it seems to be multi-threaded. I don't >> know if this will cause any trouble? > As a distro maintainer I'd always prefer option to use a library that is > already widely available. While ZSTD could certainly be added to distros, > it means the QEMU maintainer will end up having to package it & become > the defacto long term maintainer of it long term, which is an extra burden. > > WRT to making compression algorithms configurable, I think it is important > to ensure we don't add lots of optional algorithms. An important factor is > portability of images - we don't want to end up with each distro's build > of QEMU enabling a different sub-set of compression algorithms, as that is > going to harm interoperability for distributed images. This again makes me > prefer a compression format whose library is widely available, as that makes > it highly likely that the distro will choose to enable it during build. > > Regards, > Daniel Speaking about image compression - ZSDT could be questioned for different things. Though speaking about online migration - ZSTD IMHO is the best choice. I have started the investigation keeping that in mind.
Den
