----- Original Message ----- > From: "Alberto Garcia" <be...@igalia.com> > To: "qemu-discuss" <qemu-discuss@nongnu.org> > Sent: Tuesday, September 24, 2019 10:15:56 AM > Subject: Re: Large qcow2 performance
> On Mon, Sep 23, 2019 at 03:17:39PM -0500, Andrew Martin wrote: > >> I am considering creating a VM with a large qcow2, around 6TB in >> size. This Linux VM will be reading and writing a lot of small >> files to this very large qcow2 using the VirtIO driver. Are there >> any concerns or guidance about creating such a large qcow2, or >> performance issues with small files on a 6TB qcow2? > > Briefly: > > there's two important parameters that can affect performance: > cluster_size and l2_cache_size. > > cluster_size is set when you create the image, l2_cache_size is a > runtime option. > > If the L2 cache size is not large enough, I/O performance will suffer. > Check here for a summary (the defaults have changed since then, but > the basic description of the problem is correct): > > > https://blogs.igalia.com/berto/2015/12/17/improving-disk-io-performance-in-qemu-2-5-with-the-qcow2-l2-cache/ > > With small clusters you need a larger L2 cache, that is: QEMU needs > more memory in order to get the same performance. > > Images with large clusters use less memory, but there are two > drawbacks: > > - The cluster is the minimum unit of allocation, so even if you write > 512 bytes QEMU needs to allocate one complete cluster and fill it > with data. This makes *allocation* much slower the larger the > cluster size. > > See the table here for an example, but focus only on the first and > last columns and ignore the two in the middle: > > https://lists.gnu.org/archive/html/qemu-block/2019-07/msg00496.html > > After a cluster is allocated an image with large clusters should > give you the same (or possibly better) performance. > > - Because of what I just said, images with large clusters waste more > space on disk. > > A hybrid approach that combines the benefits of large clusters > regarding memory usage with the benefits of small clusters regarding > allocation performance is currently under development, and I plan to > send patches soon. > > Berto Hi Berto, Thank you for the detailed explanation! It is also worth noting that I am planning on hosting this qcow2 on a ZFS dataset (and ZFS does some in-memory caching with the ARC as well); does that have an impact on performance? Moreover, would setting the cluster_size to match the ZFS recordsize be advisable? https://jrs-s.net/2018/03/13/zvol-vs-qcow2-with-kvm/ Thanks, Andrew