Re: [PATCH v3 12/12] block/qcow2: automatically insert preallocate filter when on FUSE
19.08.2020 18:15, Stefan Hajnoczi wrote: On Mon, Aug 17, 2020 at 12:15:53PM +0300, Vladimir Sementsov-Ogievskiy wrote: vstorage has slow allocation, so this patch detect vstorage (I hope, we don't use other FUSE filesystems) and inserts preallocate filter between qcow2 node and its file child. The following test executes more than 10 times faster (43.2s -> 3.9s for me) with this patch. (/newssd3 is mount point of vstorage, both cs and mds are on same ssd local ssd disk) IMG=/newssd3/z FILE_OPTS=file.filename=$IMG COUNT=15000 CHUNK=64K CLUSTER=1M rm -f $IMG ./qemu-img create -f qcow2 -o cluster_size=$CLUSTER $IMG 1G ./qemu-img bench -c $COUNT -d 1 -s $CHUNK -w -t none -f qcow2 $IMG Kevin's input is needed here. I think the philosophy is that nodes are not automatically inserted. The user should define the graph explicitly. The management tool would be responsible for configuring a preallocate filter node. This patch originally is not intended to be merged upstream, only for downstream. I post it just to possibly get some ideas, could it be somehow useful for others. (I'm not sure, that all FUSE filesystems needs this filter. But vstorage needs) Hmm, about automatically inserted nodes: why not? block jobs insert their filters automatically.. Anyway, for our downstream process the simplest thing is to insert it automatically in Qemu. -- Best regards, Vladimir
Re: [PATCH v3 12/12] block/qcow2: automatically insert preallocate filter when on FUSE
On Mon, Aug 17, 2020 at 12:15:53PM +0300, Vladimir Sementsov-Ogievskiy wrote: > vstorage has slow allocation, so this patch detect vstorage > (I hope, we don't use other FUSE filesystems) and inserts preallocate > filter between qcow2 node and its file child. > > The following test executes more than 10 times faster > (43.2s -> 3.9s for me) with this patch. (/newssd3 is mount point of > vstorage, both cs and mds are on same ssd local ssd disk) > > IMG=/newssd3/z > FILE_OPTS=file.filename=$IMG > COUNT=15000 > CHUNK=64K > CLUSTER=1M > rm -f $IMG > ./qemu-img create -f qcow2 -o cluster_size=$CLUSTER $IMG 1G > ./qemu-img bench -c $COUNT -d 1 -s $CHUNK -w -t none -f qcow2 $IMG Kevin's input is needed here. I think the philosophy is that nodes are not automatically inserted. The user should define the graph explicitly. The management tool would be responsible for configuring a preallocate filter node. signature.asc Description: PGP signature
[PATCH v3 12/12] block/qcow2: automatically insert preallocate filter when on FUSE
vstorage has slow allocation, so this patch detect vstorage
(I hope, we don't use other FUSE filesystems) and inserts preallocate
filter between qcow2 node and its file child.
The following test executes more than 10 times faster
(43.2s -> 3.9s for me) with this patch. (/newssd3 is mount point of
vstorage, both cs and mds are on same ssd local ssd disk)
IMG=/newssd3/z
FILE_OPTS=file.filename=$IMG
COUNT=15000
CHUNK=64K
CLUSTER=1M
rm -f $IMG
./qemu-img create -f qcow2 -o cluster_size=$CLUSTER $IMG 1G
./qemu-img bench -c $COUNT -d 1 -s $CHUNK -w -t none -f qcow2 $IMG
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block/qcow2.c | 38 ++
1 file changed, 38 insertions(+)
diff --git a/block/qcow2.c b/block/qcow2.c
index 6ad6bdc166..f56507158e 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1868,6 +1868,40 @@ static void coroutine_fn qcow2_open_entry(void *opaque)
qemu_co_mutex_unlock(>lock);
}
+static int qcow2_vz_insert_prealloc_filter(BlockDriverState *bs, Error **errp)
+{
+QDict *options;
+BlockDriverState *filter_bs;
+Error *local_err = NULL;
+int flags;
+
+if (!bdrv_is_file_on_fuse(bs->file->bs)) {
+/* Nothing to do */
+return 0;
+}
+
+/* Assume it's a vstorage */
+options = qdict_new();
+qdict_put_str(options, "driver", "preallocate");
+qdict_put_str(options, "file", bs->file->bs->node_name);
+flags = bdrv_is_read_only(bs->file->bs) ? 0 : BDRV_O_RDWR;
+filter_bs = bdrv_open(NULL, NULL, options, flags, errp);
+if (!filter_bs) {
+return -EINVAL;
+}
+
+bdrv_replace_node(bs->file->bs, filter_bs, _err);
+
+/*
+ * On failure we want to remove filter_bs, on success it's referenced now
by
+ * qcow2 node.
+ */
+bdrv_unref(filter_bs);
+
+error_propagate(errp, local_err);
+return local_err ? -EINVAL : 0;
+}
+
static int qcow2_open(BlockDriverState *bs, QDict *options, int flags,
Error **errp)
{
@@ -1886,6 +1920,10 @@ static int qcow2_open(BlockDriverState *bs, QDict
*options, int flags,
return -EINVAL;
}
+if (qcow2_vz_insert_prealloc_filter(bs, errp) < 0) {
+return -EINVAL;
+}
+
/* Initialise locks */
qemu_co_mutex_init(>lock);
--
2.18.0
