Re: [PATCH 4/5] vduse-blk: Add vduse-blk resize support

2022-02-07 Thread Yongji Xie
On Mon, Feb 7, 2022 at 10:18 PM Stefan Hajnoczi wrote: > > On Tue, Jan 25, 2022 at 09:17:59PM +0800, Xie Yongji wrote: > > To support block resize, this uses vduse_dev_update_config() > > to update the capacity field in configuration space and inject > > config interrupt on the block resize

Re: [PATCH 3/5] vduse-blk: implements vduse-blk export

2022-02-07 Thread Yongji Xie
On Mon, Feb 7, 2022 at 10:15 PM Stefan Hajnoczi wrote: > > On Tue, Jan 25, 2022 at 09:17:58PM +0800, Xie Yongji wrote: > > This implements a VDUSE block backends based on > > the libvduse library. We can use it to export the BDSs > > for both VM and container (host) usage. > > > > The new

Re: [PATCH 2/5] libvduse: Add VDUSE (vDPA Device in Userspace) library

2022-02-07 Thread Yongji Xie
On Mon, Feb 7, 2022 at 10:01 PM Stefan Hajnoczi wrote: > > On Tue, Jan 25, 2022 at 09:17:57PM +0800, Xie Yongji wrote: > > VDUSE [1] is a linux framework that makes it possible to implement > > software-emulated vDPA devices in userspace. This adds a library > > as a subproject to help

Re: [PATCH 1/5] linux-headers: Add vduse.h

2022-02-07 Thread Yongji Xie
On Mon, Feb 7, 2022 at 9:12 PM Stefan Hajnoczi wrote: > > On Tue, Jan 25, 2022 at 09:17:56PM +0800, Xie Yongji wrote: > > diff --git a/scripts/update-linux-headers.sh > > b/scripts/update-linux-headers.sh > > index fea4d6eb65..4c7846076f 100755 > > --- a/scripts/update-linux-headers.sh > > +++

Re: [PATCH v8 5/5] multifd: Implement zero copy write in multifd migration (multifd-zero-copy)

2022-02-07 Thread Peter Xu
On Mon, Feb 07, 2022 at 11:49:38PM -0300, Leonardo Bras Soares Passos wrote: > Hello Peter, thanks for reviewing! > > On Mon, Feb 7, 2022 at 11:22 PM Peter Xu wrote: > > > > On Tue, Feb 01, 2022 at 03:29:03AM -0300, Leonardo Bras wrote: > > > -void multifd_send_sync_main(QEMUFile *f) > > > +int

Re: [PATCH v8 5/5] multifd: Implement zero copy write in multifd migration (multifd-zero-copy)

2022-02-07 Thread Leonardo Bras Soares Passos
Hello Peter, thanks for reviewing! On Mon, Feb 7, 2022 at 11:22 PM Peter Xu wrote: > > On Tue, Feb 01, 2022 at 03:29:03AM -0300, Leonardo Bras wrote: > > -void multifd_send_sync_main(QEMUFile *f) > > +int multifd_send_sync_main(QEMUFile *f) > > { > > int i; > > +bool flush_zero_copy; >

Re: [PATCH v8 5/5] multifd: Implement zero copy write in multifd migration (multifd-zero-copy)

2022-02-07 Thread Peter Xu
On Tue, Feb 01, 2022 at 03:29:03AM -0300, Leonardo Bras wrote: > -void multifd_send_sync_main(QEMUFile *f) > +int multifd_send_sync_main(QEMUFile *f) > { > int i; > +bool flush_zero_copy; > > if (!migrate_use_multifd()) { > -return; > +return 0; > } > if

Re: [PATCH v8 1/5] QIOChannel: Add flags on io_writev and introduce io_flush callback

2022-02-07 Thread Leonardo Bras Soares Passos
Hello Peter, On Mon, Feb 7, 2022 at 9:50 AM Peter Xu wrote: > > On Tue, Feb 01, 2022 at 03:28:59AM -0300, Leonardo Bras wrote: > > Add flags to io_writev and introduce io_flush as optional callback to > > QIOChannelClass, allowing the implementation of zero copy writes by > > subclasses. > > > >

Re: [PATCH v6 00/33] block layer: split block APIs in global state and I/O

2022-02-07 Thread Kevin Wolf
Am 21.01.2022 um 18:05 hat Emanuele Giuseppe Esposito geschrieben: > Each function in the GS API will have an assertion, checking > that it is always running under BQL. > I/O functions are instead thread safe (or so should be), meaning > that they *can* run under BQL, but also in an iothread in

Re: [PATCH v6 31/33] include/qemu/job.h: introduce job->pre_run() and use it in amend

2022-02-07 Thread Kevin Wolf
Am 21.01.2022 um 18:05 hat Emanuele Giuseppe Esposito geschrieben: > Introduce .pre_run() job callback. This cb will run in job_start, > before the coroutine is created and runs run() in the job aiocontext. > > Therefore, .pre_run() always runs in the main loop. > We can use this function

Re: [PATCH v6 09/33] block: introduce assert_bdrv_graph_writable

2022-02-07 Thread Kevin Wolf
Am 21.01.2022 um 18:05 hat Emanuele Giuseppe Esposito geschrieben: > We want to be sure that the functions that write the child and > parent list of a bs are under BQL and drain. > > BQL prevents from concurrent writings from the GS API, while > drains protect from I/O. > > TODO: drains are

Re: [RFC] thread-pool: Add option to fix the pool size

2022-02-07 Thread Stefan Hajnoczi
On Mon, Feb 07, 2022 at 01:00:46PM +0100, Nicolas Saenz Julienne wrote: > Hi Stefan, thanks for the review. I took note of your comments. > > On Thu, 2022-02-03 at 10:53 +, Stefan Hajnoczi wrote: > > > Some background on my workload: I'm using IDE emulation, the guest is an > > > old RTOS

Re: [PATCH v6 02/33] include/block/block: split header into I/O and global state API

2022-02-07 Thread Kevin Wolf
Am 01.02.2022 um 11:30 hat Paolo Bonzini geschrieben: > On 2/1/22 10:45, Emanuele Giuseppe Esposito wrote: > > > That said, even if they are a different category, I think it makes sense > > > to leave them in the same header file as I/O functions, because I/O > > > functions are locked out between

[PATCH 09/14] qcow2: qcow2_inactivate(): use qcow2_flush_caches()

2022-02-07 Thread Vladimir Sementsov-Ogievskiy
Maintaining similar logic of flushing caches in both qcow2_inactivate() and in qcow2_flush_caches() is not good. Let's refactor things to use qcow2_flush_caches() directly from qcow2_inactivate(). For this we need two things: textual error messages (add Error **) and possibility to

[PATCH 11/14] qcow2: refactor qcow2_inactivate

2022-02-07 Thread Vladimir Sementsov-Ogievskiy
Add a possibility for safe behavior: stop on first error end report in in errp. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.c | 42 -- 1 file changed, 36 insertions(+), 6 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index

Re: [PATCH v6 22/33] block/coroutines: I/O API

2022-02-07 Thread Kevin Wolf
Am 21.01.2022 um 18:05 hat Emanuele Giuseppe Esposito geschrieben: > block coroutines functions run in different aiocontext, and are > not protected by the BQL. Therefore are I/O. > > Signed-off-by: Emanuele Giuseppe Esposito > Reviewed-by: Paolo Bonzini > Reviewed-by: Stefan Hajnoczi > --- >

Re: [PATCH v6 13/33] include/block/blockjob.h: global state API

2022-02-07 Thread Kevin Wolf
Am 21.01.2022 um 18:05 hat Emanuele Giuseppe Esposito geschrieben: > blockjob functions run always under the BQL lock. > > Signed-off-by: Emanuele Giuseppe Esposito > Reviewed-by: Stefan Hajnoczi > --- > include/block/blockjob.h | 9 + > 1 file changed, 9 insertions(+) > > diff --git

[PATCH 08/14] qapi/block-core: add 'force' argument to blockdev-del

2022-02-07 Thread Vladimir Sementsov-Ogievskiy
Default behavior is force=true and it's unchanged. New behavior is force=false, which makes it possible to be sure that node removal is done successfully with no error and all metadata is stored and flushed successfully. Signed-off-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json

[PATCH 06/14] block: bdrv_delete(): drop unnecessary zeroing

2022-02-07 Thread Vladimir Sementsov-Ogievskiy
No need to zero all these things before g_free(bs). Move memory freeing to the end of the function to simplify further conversion to transaction action. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 40 ++-- 1 file changed, 14 insertions(+), 26

[PATCH 05/14] block: merge bdrv_delete and bdrv_close

2022-02-07 Thread Vladimir Sementsov-Ogievskiy
bdrv_delete() is the only caller of bdrv_close(). Let's merge them to simplify further commits. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 27 ++- 1 file changed, 10 insertions(+), 17 deletions(-) diff --git a/block.c b/block.c index

[PATCH 07/14] block: implemet bdrv_try_unref()

2022-02-07 Thread Vladimir Sementsov-Ogievskiy
Make a version of bdrv_unref() that honestly report any failure. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block.h | 1 + include/block/block_int.h | 2 + block.c | 247 +++--- 3 files changed, 208 insertions(+), 42

Re: [PATCH v6 05/33] include/sysemu/block-backend: split header into I/O and global state (GS) API

2022-02-07 Thread Kevin Wolf
Am 21.01.2022 um 18:05 hat Emanuele Giuseppe Esposito geschrieben: > Similarly to the previous patches, split block-backend.h > in block-backend-io.h and block-backend-global-state.h > > In addition, remove "block/block.h" include as it seems > it is not necessary anymore, together with

Re: [PATCH v3 1/1] util: adjust coroutine pool size to virtio block queue

2022-02-07 Thread Stefan Hajnoczi
On Fri, Jan 28, 2022 at 05:36:16PM +0900, Hiroki Narukawa wrote: > Coroutine pool size was 64 from long ago, and the basis was organized in the > commit message in c740ad92. > > At that time, virtio-blk queue-size and num-queue were not configuable, and > equivalent values were 128 and 1. > >

Re: [PATCH] block/vvfat: Fix memleaks in vvfat_close()

2022-02-07 Thread Kevin Wolf
Am 07.02.2022 um 16:20 hat Philippe Mathieu-Daudé geschrieben: > On 7/2/22 12:37, Hanna Reitz wrote: > > qcow_filename and used_clusters are allocated in enable_write_target(), > > but freed only in the error path of vvfat_open(). Free them in > > vvfat_close(), too. > > > > Signed-off-by: Hanna

[PATCH 12/14] qcow2: implement .bdrv_close_safe

2022-02-07 Thread Vladimir Sementsov-Ogievskiy
Implement new API, so that qcow2 supports blockdev-del with force=false. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block.h | 2 + block.c | 4 +- block/qcow2.c | 85 ++- 3 files changed, 72 insertions(+), 19

[PATCH 13/14] block/file-posix: implement .bdrv_close_safe

2022-02-07 Thread Vladimir Sementsov-Ogievskiy
Implement new close API for 'file' driver, so that we now have the minimal set working: qcow2 + file. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/file-posix.c | 38 +++--- 1 file changed, 31 insertions(+), 7 deletions(-) diff --git a/block/file-posix.c

[PATCH 14/14] iotests: add test for blockdev-del(force=false)

2022-02-07 Thread Vladimir Sementsov-Ogievskiy
Test for new option: use NBD server killing to simulate failure on file close. Signed-off-by: Vladimir Sementsov-Ogievskiy --- .../tests/blockdev-del-close-failure | 54 +++ .../tests/blockdev-del-close-failure.out | 4 ++ 2 files changed, 58 insertions(+) create

[PATCH 03/14] block: bdrv_refresh_perms(): allow external tran

2022-02-07 Thread Vladimir Sementsov-Ogievskiy
Allow passing external Transaction pointer, stop creating extra Transaction objects. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 31 --- 1 file changed, 20 insertions(+), 11 deletions(-) diff --git a/block.c b/block.c index 378841a546..7b65c7c5c0

[PATCH 01/14] block: refactor bdrv_remove_file_or_backing_child to bdrv_remove_child

2022-02-07 Thread Vladimir Sementsov-Ogievskiy
Now the function can remove any child, so give it more common name. Drop assertions and drop bs argument which becomes unused. Function would be reused in a further commit. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 22 -- 1 file changed, 8 insertions(+), 14

[PATCH 02/14] block: drop bdrv_detach_child()

2022-02-07 Thread Vladimir Sementsov-Ogievskiy
The only caller is bdrv_root_unref_child(), let's just do the logic directly in it. It simplifies further convertion of bdrv_root_unref_child() to transaction action. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 45 +++-- 1 file changed, 19

[PATCH 10/14] qcow2: qcow2_inactivate(): don't call qcow2_mark_clean() when RO

2022-02-07 Thread Vladimir Sementsov-Ogievskiy
qcow2_inactivate() prints errors on different failures, let's not exclude qcow2_mark_clean() call. Still, if image is read-only, no reason to report failed write and no reason even to try write. Write failure is possible when we open dirty image for check in read-only mode. Let's not do it.

[PATCH 00/14] block: blockdev-del force=false

2022-02-07 Thread Vladimir Sementsov-Ogievskiy
Hi all! Currently close-path in block layer can't fail. Neither bdrv_close() nor .bdrv_close() handlers have return value. So, when something fails (for example cached metadata flush) the only thing we can is to print an error to log. This behavior is OK on VM stop: we want to stop the running

[PATCH 04/14] block: add bdrv_try_set_aio_context_tran transaction action

2022-02-07 Thread Vladimir Sementsov-Ogievskiy
To be used in further commit. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 48 1 file changed, 48 insertions(+) diff --git a/block.c b/block.c index 7b65c7c5c0..82fbf81a3c 100644 --- a/block.c +++ b/block.c @@ -2791,6 +2791,54 @@

Re: [PATCH] hw/sd: Correct CMD58's R3 response "in idle state" bit in SPI-mode

2022-02-07 Thread Peter Maydell
On Wed, 26 Jan 2022 at 03:13, wrote: > > From: Frank Chang > > In SPI-mode, CMD58 returns R3 response with the format: > > 39 32 31 0 > ++ +---+ > | R1 | |OCR| >

Re: [PATCH] block/vvfat: Fix memleaks in vvfat_close()

2022-02-07 Thread Philippe Mathieu-Daudé via
On 7/2/22 12:37, Hanna Reitz wrote: qcow_filename and used_clusters are allocated in enable_write_target(), but freed only in the error path of vvfat_open(). Free them in vvfat_close(), too. Signed-off-by: Hanna Reitz --- block/vvfat.c | 2 ++ 1 file changed, 2 insertions(+) diff --git

Re: [PATCH] hw/sd: Correct card status clear conditions in SPI-mode

2022-02-07 Thread Peter Maydell
On Mon, 24 Jan 2022 at 06:09, wrote: > > From: Frank Chang > > In SPI-mode, unlike SD-mode, card status bits: ILLEGAL_COMMAND and > COM_CRC_ERROR have type C ("cleared by read") clear conditions. > Also, type B ("cleared on valid command") clear condition is not > supported in SPI-mode. As the

Re: [PATCH 3/5] vduse-blk: implements vduse-blk export

2022-02-07 Thread Stefan Hajnoczi
On Tue, Jan 25, 2022 at 09:17:58PM +0800, Xie Yongji wrote: > This implements a VDUSE block backends based on > the libvduse library. We can use it to export the BDSs > for both VM and container (host) usage. > > The new command-line syntax is: > > $ qemu-storage-daemon \ > --blockdev

Re: [PATCH 5/5] libvduse: Add support for reconnecting

2022-02-07 Thread Stefan Hajnoczi
On Tue, Jan 25, 2022 at 09:18:00PM +0800, Xie Yongji wrote: > To support reconnecting after restart or crash, VDUSE backend > might need to resubmit inflight I/Os. This stores the metadata > such as the index of inflight I/O's descriptors to a shm file so > that VDUSE backend can restore them

Re: [PATCH 4/5] vduse-blk: Add vduse-blk resize support

2022-02-07 Thread Stefan Hajnoczi
On Tue, Jan 25, 2022 at 09:17:59PM +0800, Xie Yongji wrote: > To support block resize, this uses vduse_dev_update_config() > to update the capacity field in configuration space and inject > config interrupt on the block resize callback. > > Signed-off-by: Xie Yongji > --- >

Re: [PATCH 2/5] libvduse: Add VDUSE (vDPA Device in Userspace) library

2022-02-07 Thread Stefan Hajnoczi
On Tue, Jan 25, 2022 at 09:17:57PM +0800, Xie Yongji wrote: > VDUSE [1] is a linux framework that makes it possible to implement > software-emulated vDPA devices in userspace. This adds a library > as a subproject to help implementing VDUSE backends in QEMU. > > [1]

Re: [PATCH 1/5] linux-headers: Add vduse.h

2022-02-07 Thread Stefan Hajnoczi
On Tue, Jan 25, 2022 at 09:17:56PM +0800, Xie Yongji wrote: > diff --git a/scripts/update-linux-headers.sh b/scripts/update-linux-headers.sh > index fea4d6eb65..4c7846076f 100755 > --- a/scripts/update-linux-headers.sh > +++ b/scripts/update-linux-headers.sh > @@ -198,6 +198,7 @@ for i in

Re: [PATCH v8 1/5] QIOChannel: Add flags on io_writev and introduce io_flush callback

2022-02-07 Thread Peter Xu
On Tue, Feb 01, 2022 at 03:28:59AM -0300, Leonardo Bras wrote: > Add flags to io_writev and introduce io_flush as optional callback to > QIOChannelClass, allowing the implementation of zero copy writes by > subclasses. > > How to use them: > - Write data using

Re: [RFC] thread-pool: Add option to fix the pool size

2022-02-07 Thread Nicolas Saenz Julienne
Hi Stefan, thanks for the review. I took note of your comments. On Thu, 2022-02-03 at 10:53 +, Stefan Hajnoczi wrote: > > Some background on my workload: I'm using IDE emulation, the guest is an > > old RTOS that doesn't support virtio, using 'aio=native' isn't possible > > either (unaligned

[PATCH] block/vvfat: Fix memleaks in vvfat_close()

2022-02-07 Thread Hanna Reitz
qcow_filename and used_clusters are allocated in enable_write_target(), but freed only in the error path of vvfat_open(). Free them in vvfat_close(), too. Signed-off-by: Hanna Reitz --- block/vvfat.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/block/vvfat.c b/block/vvfat.c index