[PATCH v3 5/6] block/linux-aio: convert to blk_io_plug_call() API

2023-05-30 Thread Stefan Hajnoczi
to laio_unplug_fn(). It is not obvious that this condition affects performance in practice, so I am removing it instead of trying to come up with a more complex mechanism to preserve the condition. Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake --- include/block/raw-aio.h | 7 --- block

[PATCH v3 1/6] block: add blk_io_plug_call() API

2023-05-30 Thread Stefan Hajnoczi
()/blk_io_unplug() no longer require a BlockBackend argument because the plug state is now thread-local. Later patches convert block drivers to blk_io_plug_call() and then we can finally remove .bdrv_co_io_plug() once all block drivers have been converted. Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake

[PATCH v3 4/6] block/io_uring: convert to blk_io_plug_call() API

2023-05-30 Thread Stefan Hajnoczi
Stop using the .bdrv_co_io_plug() API because it is not multi-queue block layer friendly. Use the new blk_io_plug_call() API to batch I/O submission instead. Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake Reviewed-by: Stefano Garzarella --- v2 - Removed whitespace hunk [Eric

[PATCH v3 6/6] block: remove bdrv_co_io_plug() API

2023-05-30 Thread Stefan Hajnoczi
No block driver implements .bdrv_co_io_plug() anymore. Get rid of the function pointers. Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake Reviewed-by: Stefano Garzarella --- include/block/block-io.h | 3 --- include/block/block_int-common.h | 11 -- block/io.c

[PATCH v3 3/6] block/blkio: convert to blk_io_plug_call() API

2023-05-30 Thread Stefan Hajnoczi
Stop using the .bdrv_co_io_plug() API because it is not multi-queue block layer friendly. Use the new blk_io_plug_call() API to batch I/O submission instead. Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake Reviewed-by: Stefano Garzarella --- v2 - Add missing #include and fix

[PATCH v3 2/6] block/nvme: convert to blk_io_plug_call() API

2023-05-30 Thread Stefan Hajnoczi
Stop using the .bdrv_co_io_plug() API because it is not multi-queue block layer friendly. Use the new blk_io_plug_call() API to batch I/O submission instead. Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake Reviewed-by: Stefano Garzarella --- v2 - Remove unused

[PATCH v3 0/6] block: add blk_io_plug_call() API

2023-05-30 Thread Stefan Hajnoczi
and a Jupyter notebook are available here: https://github.com/stefanha/qemu-perf/tree/remove-blk_io_plug Stefan Hajnoczi (6): block: add blk_io_plug_call() API block/nvme: convert to blk_io_plug_call() API block/blkio: convert to blk_io_plug_call() API block/io_uring: convert to blk_io_plug_c

Re: [PATCH v2 5/6] block/linux-aio: convert to blk_io_plug_call() API

2023-05-30 Thread Stefan Hajnoczi
On Mon, May 29, 2023 at 10:50:34AM +0200, Stefano Garzarella wrote: > On Wed, May 24, 2023 at 03:36:34PM -0400, Stefan Hajnoczi wrote: > > On Wed, May 24, 2023 at 10:52:03AM +0200, Stefano Garzarella wrote: > > > On Tue, May 23, 2023 at 01:12:59PM -0400, Stefan Hajnoczi wrote:

Re: [PATCH v2 5/6] block/linux-aio: convert to blk_io_plug_call() API

2023-05-25 Thread Stefan Hajnoczi
On Wed, May 24, 2023 at 10:52:03AM +0200, Stefano Garzarella wrote: > On Tue, May 23, 2023 at 01:12:59PM -0400, Stefan Hajnoczi wrote: > > Stop using the .bdrv_co_io_plug() API because it is not multi-queue > > block layer friendly. Use the new blk_io_plug_call() API to batch I/O

[PATCH v2 5/6] block/linux-aio: convert to blk_io_plug_call() API

2023-05-23 Thread Stefan Hajnoczi
Stop using the .bdrv_co_io_plug() API because it is not multi-queue block layer friendly. Use the new blk_io_plug_call() API to batch I/O submission instead. Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake --- include/block/raw-aio.h | 7 --- block/file-posix.c | 28

[PATCH v2 6/6] block: remove bdrv_co_io_plug() API

2023-05-23 Thread Stefan Hajnoczi
No block driver implements .bdrv_co_io_plug() anymore. Get rid of the function pointers. Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake --- include/block/block-io.h | 3 --- include/block/block_int-common.h | 11 -- block/io.c | 37

[PATCH v2 3/6] block/blkio: convert to blk_io_plug_call() API

2023-05-23 Thread Stefan Hajnoczi
Stop using the .bdrv_co_io_plug() API because it is not multi-queue block layer friendly. Use the new blk_io_plug_call() API to batch I/O submission instead. Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake --- v2 - Add missing #include and fix blkio_unplug_fn() prototype [Stefano

[PATCH v2 1/6] block: add blk_io_plug_call() API

2023-05-23 Thread Stefan Hajnoczi
()/blk_io_unplug() no longer require a BlockBackend argument because the plug state is now thread-local. Later patches convert block drivers to blk_io_plug_call() and then we can finally remove .bdrv_co_io_plug() once all block drivers have been converted. Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake

[PATCH v2 2/6] block/nvme: convert to blk_io_plug_call() API

2023-05-23 Thread Stefan Hajnoczi
Stop using the .bdrv_co_io_plug() API because it is not multi-queue block layer friendly. Use the new blk_io_plug_call() API to batch I/O submission instead. Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake --- v2 - Remove unused nvme_process_completion_queue_plugged trace event

[PATCH v2 4/6] block/io_uring: convert to blk_io_plug_call() API

2023-05-23 Thread Stefan Hajnoczi
Stop using the .bdrv_co_io_plug() API because it is not multi-queue block layer friendly. Use the new blk_io_plug_call() API to batch I/O submission instead. Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric Blake --- v2 - Removed whitespace hunk [Eric] --- include/block/raw-aio.h | 7

[PATCH v2 0/6] block: add blk_io_plug_call() API

2023-05-23 Thread Stefan Hajnoczi
a/qemu-perf/tree/remove-blk_io_plug Stefan Hajnoczi (6): block: add blk_io_plug_call() API block/nvme: convert to blk_io_plug_call() API block/blkio: convert to blk_io_plug_call() API block/io_uring: convert to blk_io_plug_call() API block/linux-aio: convert to blk_io_plug_call() API block:

Re: [PATCH 3/6] block/blkio: convert to blk_io_plug_call() API

2023-05-23 Thread Stefan Hajnoczi
On Fri, May 19, 2023 at 10:47:00AM +0200, Stefano Garzarella wrote: > On Wed, May 17, 2023 at 06:10:19PM -0400, Stefan Hajnoczi wrote: > > Stop using the .bdrv_co_io_plug() API because it is not multi-queue > > block layer friendly. Use the new blk_io_plug_call() API to batch I/O

Re: [PATCH 4/6] block/io_uring: convert to blk_io_plug_call() API

2023-05-23 Thread Stefan Hajnoczi
On Thu, May 18, 2023 at 07:18:42PM -0500, Eric Blake wrote: > On Wed, May 17, 2023 at 06:10:20PM -0400, Stefan Hajnoczi wrote: > > Stop using the .bdrv_co_io_plug() API because it is not multi-queue > > block layer friendly. Use the new blk_io_plug_call() API to batch I/O > &g

Re: [PATCH 1/6] block: add blk_io_plug_call() API

2023-05-23 Thread Stefan Hajnoczi
On Thu, May 18, 2023 at 07:04:52PM -0500, Eric Blake wrote: > On Wed, May 17, 2023 at 06:10:17PM -0400, Stefan Hajnoczi wrote: > > Introduce a new API for thread-local blk_io_plug() that does not > > traverse the block graph. The goal is to make blk_io_plug() multi-que

Re: [PATCH 2/6] block/nvme: convert to blk_io_plug_call() API

2023-05-23 Thread Stefan Hajnoczi
On Fri, May 19, 2023 at 10:46:25AM +0200, Stefano Garzarella wrote: > On Wed, May 17, 2023 at 06:10:18PM -0400, Stefan Hajnoczi wrote: > > Stop using the .bdrv_co_io_plug() API because it is not multi-queue > > block layer friendly. Use the new blk_io_plug_call() API to batch I/O

Re: [PATCH 1/6] block: add blk_io_plug_call() API

2023-05-23 Thread Stefan Hajnoczi
On Fri, May 19, 2023 at 10:45:57AM +0200, Stefano Garzarella wrote: > On Wed, May 17, 2023 at 06:10:17PM -0400, Stefan Hajnoczi wrote: > > Introduce a new API for thread-local blk_io_plug() that does not > > traverse the block graph. The goal is to make blk_io_plug() multi-que

[PATCH 0/6] block: add blk_io_plug_call() API

2023-05-17 Thread Stefan Hajnoczi
/stefanha/qemu-perf/tree/remove-blk_io_plug Stefan Hajnoczi (6): block: add blk_io_plug_call() API block/nvme: convert to blk_io_plug_call() API block/blkio: convert to blk_io_plug_call() API block/io_uring: convert to blk_io_plug_call() API block/linux-aio: convert to blk_io_plug_call

[PATCH 4/6] block/io_uring: convert to blk_io_plug_call() API

2023-05-17 Thread Stefan Hajnoczi
Stop using the .bdrv_co_io_plug() API because it is not multi-queue block layer friendly. Use the new blk_io_plug_call() API to batch I/O submission instead. Signed-off-by: Stefan Hajnoczi --- include/block/raw-aio.h | 7 --- block/file-posix.c | 10 - block/io_uring.c

[PATCH 5/6] block/linux-aio: convert to blk_io_plug_call() API

2023-05-17 Thread Stefan Hajnoczi
Stop using the .bdrv_co_io_plug() API because it is not multi-queue block layer friendly. Use the new blk_io_plug_call() API to batch I/O submission instead. Signed-off-by: Stefan Hajnoczi --- include/block/raw-aio.h | 7 --- block/file-posix.c | 28 block

[PATCH 6/6] block: remove bdrv_co_io_plug() API

2023-05-17 Thread Stefan Hajnoczi
No block driver implements .bdrv_co_io_plug() anymore. Get rid of the function pointers. Signed-off-by: Stefan Hajnoczi --- include/block/block-io.h | 3 --- include/block/block_int-common.h | 11 -- block/io.c | 37 3

[PATCH 1/6] block: add blk_io_plug_call() API

2023-05-17 Thread Stefan Hajnoczi
()/blk_io_unplug() no longer require a BlockBackend argument because the plug state is now thread-local. Later patches convert block drivers to blk_io_plug_call() and then we can finally remove .bdrv_co_io_plug() once all block drivers have been converted. Signed-off-by: Stefan Hajnoczi --- MAINTAINERS

[PATCH 3/6] block/blkio: convert to blk_io_plug_call() API

2023-05-17 Thread Stefan Hajnoczi
Stop using the .bdrv_co_io_plug() API because it is not multi-queue block layer friendly. Use the new blk_io_plug_call() API to batch I/O submission instead. Signed-off-by: Stefan Hajnoczi --- block/blkio.c | 40 +--- 1 file changed, 21 insertions(+), 19

[PATCH 2/6] block/nvme: convert to blk_io_plug_call() API

2023-05-17 Thread Stefan Hajnoczi
Stop using the .bdrv_co_io_plug() API because it is not multi-queue block layer friendly. Use the new blk_io_plug_call() API to batch I/O submission instead. Signed-off-by: Stefan Hajnoczi --- block/nvme.c | 44 1 file changed, 12 insertions(+), 32

[PATCH v6 17/20] virtio-blk: implement BlockDevOps->drained_begin()

2023-05-16 Thread Stefan Hajnoczi
attaching/detaching ioeventfds if the data plane is started/stopped during a drained section. This should be rare, but maybe the mirror block job can trigger it. Signed-off-by: Stefan Hajnoczi --- hw/block/dataplane/virtio-blk.c | 16 -- hw/block/virtio-blk.c | 38

[PATCH v6 18/20] virtio-scsi: implement BlockDevOps->drained_begin()

2023-05-16 Thread Stefan Hajnoczi
sees one pair of .drained_begin/end() calls. After this commit, virtio-scsi no longer depends on hw/virtio's ioeventfd aio_set_event_notifier(is_external=true). This commit is a step towards removing the aio_disable_external() API. Signed-off-by: Stefan Hajnoczi --- include/hw/scsi/scsi.h

[PATCH v6 16/20] virtio: make it possible to detach host notifier from any thread

2023-05-16 Thread Stefan Hajnoczi
of virtio_queue_aio_detach_host_notifier() so that the function can be called from any thread. This is in preparation for the next patch. Signed-off-by: Stefan Hajnoczi --- hw/block/dataplane/virtio-blk.c | 7 +++ hw/scsi/virtio-scsi-dataplane.c | 14 ++ hw/virtio/virtio.c

[PATCH v6 12/20] hw/xen: do not set is_external=true on evtchn fds

2023-05-16 Thread Stefan Hajnoczi
BlockDevOps .drained_begin/end() callbacks. It no longer relies on is_external=true so it is safe to pass is_external=false. This is part of ongoing work to remove the aio_disable_external() API. Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf --- hw/xen/xen-bus.c | 8 1 file

[PATCH v6 11/20] xen-block: implement BlockDevOps->drained_begin()

2023-05-16 Thread Stefan Hajnoczi
on the dataplane instance, so move the blk_set_dev_ops() call after xen_block_dataplane_create(). Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf --- hw/block/dataplane/xen-block.h | 2 ++ hw/block/dataplane/xen-block.c | 42 +- hw/block/xen-block.c

[PATCH v6 20/20] aio: remove aio_disable_external() API

2023-05-16 Thread Stefan Hajnoczi
, is_external, io_read, io_poll, io_poll_ready; @@ - aio_set_event_notifier(ctx, notifier, is_external, io_read, io_poll, io_poll_ready) + aio_set_event_notifier(ctx, notifier, io_read, io_poll, io_poll_ready) Reviewed-by: Juan Quintela Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Stefan

[PATCH v6 19/20] virtio: do not set is_external=true on host notifiers

2023-05-16 Thread Stefan Hajnoczi
Host notifiers can now use is_external=false since virtio-blk and virtio-scsi no longer rely on is_external=true for drained sections. Signed-off-by: Stefan Hajnoczi --- hw/virtio/virtio.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio

[PATCH v6 13/20] block/export: rewrite vduse-blk drain code

2023-05-16 Thread Stefan Hajnoczi
in .drained_end(). This eliminates the need for the aio_set_fd_handler(is_external=true) flag, which is being removed from QEMU. This is a long list but splitting it into individual commits would probably lead to git bisect failures - the changes are all related. Signed-off-by: Stefan Hajnoczi

[PATCH v6 15/20] block/fuse: do not set is_external=true on FUSE fd

2023-05-16 Thread Stefan Hajnoczi
This is part of ongoing work to remove the aio_disable_external() API. Use BlockDevOps .drained_begin/end/poll() instead of aio_set_fd_handler(is_external=true). As a side-effect the FUSE export now follows AioContext changes like the other export types. Signed-off-by: Stefan Hajnoczi Reviewed

[PATCH v6 14/20] block/export: don't require AioContext lock around blk_exp_ref/unref()

2023-05-16 Thread Stefan Hajnoczi
The FUSE export calls blk_exp_ref/unref() without the AioContext lock. Instead of fixing the FUSE export, adjust blk_exp_ref/unref() so they work without the AioContext lock. This way it's less error-prone. Suggested-by: Paolo Bonzini Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf

[PATCH v6 10/20] block: drain from main loop thread in bdrv_co_yield_to_drain()

2023-05-16 Thread Stefan Hajnoczi
, and BlockDriver. Narrow IO_OR_GS_CODE() to GLOBAL_STATE_CODE() where appropriate. The test-bdrv-drain test case calls bdrv_drain() from an IOThread. This is now only allowed from coroutine context, so update the test case to run in a coroutine. Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin

[PATCH v6 09/20] block: add blk_in_drain() API

2023-05-16 Thread Stefan Hajnoczi
The BlockBackend quiesce_counter is greater than zero during drained sections. Add an API to check whether the BlockBackend is in a drained section. The next patch will use this API. Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf --- include/sysemu/block-backend-global-state.h | 1

[PATCH v6 08/20] hw/xen: do not use aio_set_fd_handler(is_external=true) in xen_xenstore

2023-05-16 Thread Stefan Hajnoczi
-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf --- hw/i386/kvm/xen_xenstore.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/i386/kvm/xen_xenstore.c b/hw/i386/kvm/xen_xenstore.c index 900679af8a..6e81bc8791 100644 --- a/hw/i386/kvm/xen_xenstore.c +++ b/hw/i386/kvm

[PATCH v6 06/20] block/export: wait for vhost-user-blk requests when draining

2023-05-16 Thread Stefan Hajnoczi
there are requests in flight. The in-flight counter needs to be atomic. Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf --- v5: - Use atomic accesses for in_flight counter in vhost-user-server.c [Kevin] --- include/qemu/vhost-user-server.h | 4 +++- block/export/vhost-user-blk-server.c

[PATCH v6 07/20] block/export: stop using is_external in vhost-user-blk server

2023-05-16 Thread Stefan Hajnoczi
the aio_disable_external() API cannot be used since multiple AioContext may be processing I/O, not just one. Switch to BlockDevOps->drained_begin/end() callbacks. Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf --- block/export/vhost-user-blk-server.c | 28 ++-- util/vhost-u

[PATCH v6 05/20] util/vhost-user-server: rename refcount to in_flight counter

2023-05-16 Thread Stefan Hajnoczi
there are no more requests. Avoid confusing by renaming refcount and ref/unref to in_flight and inc/dec. Reviewed-by: Paolo Bonzini Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf --- include/qemu/vhost-user-server.h | 6 +++--- block/export/vhost-user

[PATCH v6 04/20] virtio-scsi: stop using aio_disable_external() during unplug

2023-05-16 Thread Stefan Hajnoczi
equests remain once qdev_simple_device_unplug_cb() returns. Thanks to these two conditions we don't need aio_disable_external() anymore. Cc: Zhengui Li Reviewed-by: Paolo Bonzini Reviewed-by: Daniil Tatianin Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf --- hw/scsi/virtio-scsi.c | 3 --- 1 file c

[PATCH v6 03/20] virtio-scsi: avoid race between unplug and transport event

2023-05-16 Thread Stefan Hajnoczi
. These changes ensure that the guest driver does not see the SCSIDevice that's being unplugged if it responds very quickly to the transport reset event. Reviewed-by: Paolo Bonzini Reviewed-by: Michael S. Tsirkin Reviewed-by: Daniil Tatianin Signed-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf

[PATCH v6 02/20] hw/qdev: introduce qdev_is_realized() helper

2023-05-16 Thread Stefan Hajnoczi
-off-by: Stefan Hajnoczi Reviewed-by: Kevin Wolf --- include/hw/qdev-core.h | 17 ++--- hw/scsi/scsi-bus.c | 3 +-- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h index 7623703943..f1070d6dc7 100644 --- a/include/hw

[PATCH v6 01/20] block-backend: split blk_do_set_aio_context()

2023-05-16 Thread Stefan Hajnoczi
o_ctx_commit() being invoked as part of the AioContext change propagation. This can be solved by temporarily setting blk->allow_aio_context_change to true. Future patches call blk_get_aio_context() from BlockDevOps->drained_end(), so this patch will become necessary. Signed-off-by: Stefan

[PATCH v6 00/20] block: remove aio_disable_external() API

2023-05-16 Thread Stefan Hajnoczi
file descriptor handlers. This ensures that new I/O requests are not submitted in drained sections. Stefan Hajnoczi (20): block-backend: split blk_do_set_aio_context() hw/qdev: introduce qdev_is_realized() helper virtio-scsi: avoid race between unplug and transport event virtio-

Re: [PATCH v4 20/20] aio: remove aio_disable_external() API

2023-05-11 Thread Stefan Hajnoczi
On Thu, May 04, 2023 at 11:34:17PM +0200, Kevin Wolf wrote: > Am 25.04.2023 um 19:27 hat Stefan Hajnoczi geschrieben: > > All callers now pass is_external=false to aio_set_fd_handler() and > > aio_set_event_notifier(). The aio_disable_external() API that > > temporarily

Re: [PATCH v4 17/20] virtio-blk: implement BlockDevOps->drained_begin()

2023-05-11 Thread Stefan Hajnoczi
On Thu, May 04, 2023 at 11:13:42PM +0200, Kevin Wolf wrote: > Am 25.04.2023 um 19:27 hat Stefan Hajnoczi geschrieben: > > Detach ioeventfds during drained sections to stop I/O submission from > > the guest. virtio-blk is no longer reliant on aio_disable_external() >

Re: [PATCH v4 17/20] virtio-blk: implement BlockDevOps->drained_begin()

2023-05-11 Thread Stefan Hajnoczi
On Thu, May 04, 2023 at 11:13:42PM +0200, Kevin Wolf wrote: > Am 25.04.2023 um 19:27 hat Stefan Hajnoczi geschrieben: > > Detach ioeventfds during drained sections to stop I/O submission from > > the guest. virtio-blk is no longer reliant on aio_disable_external() >

Re: [PATCH v4 16/20] virtio: make it possible to detach host notifier from any thread

2023-05-11 Thread Stefan Hajnoczi
On Thu, May 04, 2023 at 11:00:35PM +0200, Kevin Wolf wrote: > Am 25.04.2023 um 19:27 hat Stefan Hajnoczi geschrieben: > > virtio_queue_aio_detach_host_notifier() does two things: > > 1. It removes the fd handler from the event loop. > > 2. It processes the vi

Re: [PATCH v5 05/21] virtio-scsi: stop using aio_disable_external() during unplug

2023-05-09 Thread Stefan Hajnoczi
On Tue, May 09, 2023 at 08:55:14PM +0200, Kevin Wolf wrote: > Am 04.05.2023 um 21:53 hat Stefan Hajnoczi geschrieben: > > This patch is part of an effort to remove the aio_disable_external() > > API because it does not fit in a multi-queue block layer world where > >

Re: [PATCH v5 00/21] block: remove aio_disable_external() API

2023-05-09 Thread Stefan Hajnoczi
On Thu, May 04, 2023 at 11:44:42PM +0200, Kevin Wolf wrote: > Am 04.05.2023 um 21:53 hat Stefan Hajnoczi geschrieben: > > v5: > > - Use atomic accesses for in_flight counter in vhost-user-server.c [Kevin] > > - Stash SCSIDevice id/lun values for VIRTIO_SCSI_T_TRANSPORT_RES

[PATCH v5 13/21] hw/xen: do not set is_external=true on evtchn fds

2023-05-04 Thread Stefan Hajnoczi
BlockDevOps .drained_begin/end() callbacks. It no longer relies on is_external=true so it is safe to pass is_external=false. This is part of ongoing work to remove the aio_disable_external() API. Signed-off-by: Stefan Hajnoczi --- hw/xen/xen-bus.c | 8 1 file changed, 4 insertions(+), 4

[PATCH v5 15/21] block/export: don't require AioContext lock around blk_exp_ref/unref()

2023-05-04 Thread Stefan Hajnoczi
The FUSE export calls blk_exp_ref/unref() without the AioContext lock. Instead of fixing the FUSE export, adjust blk_exp_ref/unref() so they work without the AioContext lock. This way it's less error-prone. Suggested-by: Paolo Bonzini Signed-off-by: Stefan Hajnoczi --- include/block/export.h

[PATCH v5 21/21] aio: remove aio_disable_external() API

2023-05-04 Thread Stefan Hajnoczi
, is_external, io_read, io_poll, io_poll_ready; @@ - aio_set_event_notifier(ctx, notifier, is_external, io_read, io_poll, io_poll_ready) + aio_set_event_notifier(ctx, notifier, io_read, io_poll, io_poll_ready) Reviewed-by: Juan Quintela Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Stefan

[PATCH v5 17/21] virtio: make it possible to detach host notifier from any thread

2023-05-04 Thread Stefan Hajnoczi
-by: Stefan Hajnoczi --- hw/block/dataplane/virtio-blk.c | 2 ++ hw/scsi/virtio-scsi-dataplane.c | 9 + 2 files changed, 11 insertions(+) diff --git a/hw/block/dataplane/virtio-blk.c b/hw/block/dataplane/virtio-blk.c index a6202997ee..27eafa6c92 100644 --- a/hw/block/dataplane/virtio-blk.c +++ b

[PATCH v5 12/21] xen-block: implement BlockDevOps->drained_begin()

2023-05-04 Thread Stefan Hajnoczi
on the dataplane instance, so move the blk_set_dev_ops() call after xen_block_dataplane_create(). Signed-off-by: Stefan Hajnoczi --- hw/block/dataplane/xen-block.h | 2 ++ hw/block/dataplane/xen-block.c | 42 +- hw/block/xen-block.c | 24

[PATCH v5 19/21] virtio-scsi: implement BlockDevOps->drained_begin()

2023-05-04 Thread Stefan Hajnoczi
sees one pair of .drained_begin/end() calls. After this commit, virtio-scsi no longer depends on hw/virtio's ioeventfd aio_set_event_notifier(is_external=true). This commit is a step towards removing the aio_disable_external() API. Signed-off-by: Stefan Hajnoczi --- include/hw/scsi/scsi.h

[PATCH v5 18/21] virtio-blk: implement BlockDevOps->drained_begin()

2023-05-04 Thread Stefan Hajnoczi
attaching/detaching ioeventfds if the data plane is started/stopped during a drained section. This should be rare, but maybe the mirror block job can trigger it. Signed-off-by: Stefan Hajnoczi --- hw/block/dataplane/virtio-blk.c | 17 +-- hw/block/virtio-blk.c | 38

[PATCH v5 14/21] block/export: rewrite vduse-blk drain code

2023-05-04 Thread Stefan Hajnoczi
in .drained_end(). This eliminates the need for the aio_set_fd_handler(is_external=true) flag, which is being removed from QEMU. This is a long list but splitting it into individual commits would probably lead to git bisect failures - the changes are all related. Signed-off-by: Stefan Hajnoczi

[PATCH v5 20/21] virtio: do not set is_external=true on host notifiers

2023-05-04 Thread Stefan Hajnoczi
Host notifiers can now use is_external=false since virtio-blk and virtio-scsi no longer rely on is_external=true for drained sections. Signed-off-by: Stefan Hajnoczi --- hw/virtio/virtio.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio

[PATCH v5 16/21] block/fuse: do not set is_external=true on FUSE fd

2023-05-04 Thread Stefan Hajnoczi
This is part of ongoing work to remove the aio_disable_external() API. Use BlockDevOps .drained_begin/end/poll() instead of aio_set_fd_handler(is_external=true). As a side-effect the FUSE export now follows AioContext changes like the other export types. Signed-off-by: Stefan Hajnoczi

[PATCH v5 11/21] block: drain from main loop thread in bdrv_co_yield_to_drain()

2023-05-04 Thread Stefan Hajnoczi
, and BlockDriver. Narrow IO_OR_GS_CODE() to GLOBAL_STATE_CODE() where appropriate. The test-bdrv-drain test case calls bdrv_drain() from an IOThread. This is now only allowed from coroutine context, so update the test case to run in a coroutine. Signed-off-by: Stefan Hajnoczi --- include/block

[PATCH v5 10/21] block: add blk_in_drain() API

2023-05-04 Thread Stefan Hajnoczi
The BlockBackend quiesce_counter is greater than zero during drained sections. Add an API to check whether the BlockBackend is in a drained section. The next patch will use this API. Signed-off-by: Stefan Hajnoczi --- include/sysemu/block-backend-global-state.h | 1 + block/block-backend.c

[PATCH v5 08/21] block/export: stop using is_external in vhost-user-blk server

2023-05-04 Thread Stefan Hajnoczi
the aio_disable_external() API cannot be used since multiple AioContext may be processing I/O, not just one. Switch to BlockDevOps->drained_begin/end() callbacks. Signed-off-by: Stefan Hajnoczi --- block/export/vhost-user-blk-server.c | 28 ++-- util/vhost-user-server.c |

[PATCH v5 09/21] hw/xen: do not use aio_set_fd_handler(is_external=true) in xen_xenstore

2023-05-04 Thread Stefan Hajnoczi
-by: Stefan Hajnoczi --- hw/i386/kvm/xen_xenstore.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/i386/kvm/xen_xenstore.c b/hw/i386/kvm/xen_xenstore.c index 900679af8a..6e81bc8791 100644 --- a/hw/i386/kvm/xen_xenstore.c +++ b/hw/i386/kvm/xen_xenstore.c @@ -133,7 +133,7

[PATCH v5 07/21] block/export: wait for vhost-user-blk requests when draining

2023-05-04 Thread Stefan Hajnoczi
there are requests in flight. The in-flight counter needs to be atomic. Signed-off-by: Stefan Hajnoczi --- v5: - Use atomic accesses for in_flight counter in vhost-user-server.c [Kevin] --- include/qemu/vhost-user-server.h | 4 +++- block/export/vhost-user-blk-server.c | 13 + util

[PATCH v5 06/21] util/vhost-user-server: rename refcount to in_flight counter

2023-05-04 Thread Stefan Hajnoczi
there are no more requests. Avoid confusing by renaming refcount and ref/unref to in_flight and inc/dec. Reviewed-by: Paolo Bonzini Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Stefan Hajnoczi --- include/qemu/vhost-user-server.h | 6 +++--- block/export/vhost-user-blk-server.c | 11

[PATCH v5 04/21] virtio-scsi: avoid race between unplug and transport event

2023-05-04 Thread Stefan Hajnoczi
. These changes ensure that the guest driver does not see the SCSIDevice that's being unplugged if it responds very quickly to the transport reset event. Reviewed-by: Paolo Bonzini Reviewed-by: Michael S. Tsirkin Reviewed-by: Daniil Tatianin Signed-off-by: Stefan Hajnoczi --- v5: - Stash

[PATCH v5 05/21] virtio-scsi: stop using aio_disable_external() during unplug

2023-05-04 Thread Stefan Hajnoczi
equests remain once qdev_simple_device_unplug_cb() returns. Thanks to these two conditions we don't need aio_disable_external() anymore. Cc: Zhengui Li Reviewed-by: Paolo Bonzini Reviewed-by: Daniil Tatianin Signed-off-by: Stefan Hajnoczi --- hw/scsi/virtio-scsi.c | 3 --- 1 file changed, 3 deletions(-) d

[PATCH v5 03/21] hw/qdev: introduce qdev_is_realized() helper

2023-05-04 Thread Stefan Hajnoczi
-off-by: Stefan Hajnoczi --- include/hw/qdev-core.h | 17 ++--- hw/scsi/scsi-bus.c | 3 +-- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h index 7623703943..f1070d6dc7 100644 --- a/include/hw/qdev-core.h +++ b/include

[PATCH v5 00/21] block: remove aio_disable_external() API

2023-05-04 Thread Stefan Hajnoczi
I/O requests are not submitted in drained sections. Kevin Wolf (1): block: Fix use after free in blockdev_mark_auto_del() Stefan Hajnoczi (20): block-backend: split blk_do_set_aio_context() hw/qdev: introduce qdev_is_realized() helper virtio-scsi: avoid race between unplug and transport event

[PATCH v5 02/21] block-backend: split blk_do_set_aio_context()

2023-05-04 Thread Stefan Hajnoczi
o_ctx_commit() being invoked as part of the AioContext change propagation. This can be solved by temporarily setting blk->allow_aio_context_change to true. Future patches call blk_get_aio_context() from BlockDevOps->drained_end(), so this patch will become necessary. Signed-off-by: Stefan Hajnoczi

[PATCH v5 01/21] block: Fix use after free in blockdev_mark_auto_del()

2023-05-04 Thread Stefan Hajnoczi
. For both reasons, we can't just continue at block_job_next_locked(job). Instead, start at the head of the list again after job_cancel_locked() and skip those jobs that we already cancelled (or that are completing anyway). Signed-off-by: Kevin Wolf Reviewed-by: Stefan Hajnoczi Signed-off-by: Stefan

Re: [PATCH v4 03/20] virtio-scsi: avoid race between unplug and transport event

2023-05-03 Thread Stefan Hajnoczi
On Wed, May 03, 2023 at 10:00:57AM +0200, Kevin Wolf wrote: > Am 02.05.2023 um 20:56 hat Stefan Hajnoczi geschrieben: > > On Tue, May 02, 2023 at 05:19:46PM +0200, Kevin Wolf wrote: > > > Am 25.04.2023 um 19:26 hat Stefan Hajnoczi geschrieben: > > > > Only

Re: [PATCH v4 07/20] block/export: stop using is_external in vhost-user-blk server

2023-05-03 Thread Stefan Hajnoczi
On Wed, May 03, 2023 at 10:08:46AM +0200, Kevin Wolf wrote: > Am 02.05.2023 um 22:06 hat Stefan Hajnoczi geschrieben: > > On Tue, May 02, 2023 at 06:04:24PM +0200, Kevin Wolf wrote: > > > Am 25.04.2023 um 19:27 hat Stefan Hajnoczi geschrieben: > > > > vhost-user act

Re: [PATCH v4 04/20] virtio-scsi: stop using aio_disable_external() during unplug

2023-05-03 Thread Stefan Hajnoczi
On Wed, May 03, 2023 at 01:40:49PM +0200, Kevin Wolf wrote: > Am 02.05.2023 um 22:02 hat Stefan Hajnoczi geschrieben: > > On Tue, May 02, 2023 at 03:19:52PM +0200, Kevin Wolf wrote: > > > Am 01.05.2023 um 17:09 hat Stefan Hajnoczi geschrieben: > > > > On Fri, Ap

Re: [PATCH v4 10/20] block: drain from main loop thread in bdrv_co_yield_to_drain()

2023-05-02 Thread Stefan Hajnoczi
On Tue, May 02, 2023 at 06:21:20PM +0200, Kevin Wolf wrote: > Am 25.04.2023 um 19:27 hat Stefan Hajnoczi geschrieben: > > For simplicity, always run BlockDevOps .drained_begin/end/poll() > > callbacks in the main loop thread. This makes it easier to implement the > > callb

Re: [PATCH v4 07/20] block/export: stop using is_external in vhost-user-blk server

2023-05-02 Thread Stefan Hajnoczi
On Tue, May 02, 2023 at 06:04:24PM +0200, Kevin Wolf wrote: > Am 25.04.2023 um 19:27 hat Stefan Hajnoczi geschrieben: > > vhost-user activity must be suspended during bdrv_drained_begin/end(). > > This prevents new requests from interfering with whatever is happening > > i

Re: [PATCH v4 04/20] virtio-scsi: stop using aio_disable_external() during unplug

2023-05-02 Thread Stefan Hajnoczi
On Tue, May 02, 2023 at 03:19:52PM +0200, Kevin Wolf wrote: > Am 01.05.2023 um 17:09 hat Stefan Hajnoczi geschrieben: > > On Fri, Apr 28, 2023 at 04:22:55PM +0200, Kevin Wolf wrote: > > > Am 25.04.2023 um 19:27 hat Stefan Hajnoczi geschrieben: > > > > This patch

Re: [PATCH v4 06/20] block/export: wait for vhost-user-blk requests when draining

2023-05-02 Thread Stefan Hajnoczi
On Tue, May 02, 2023 at 05:42:51PM +0200, Kevin Wolf wrote: > Am 25.04.2023 um 19:27 hat Stefan Hajnoczi geschrieben: > > Each vhost-user-blk request runs in a coroutine. When the BlockBackend > > enters a drained section we need to enter a quiescent state. Currently > > a

Re: [PATCH v4 03/20] virtio-scsi: avoid race between unplug and transport event

2023-05-02 Thread Stefan Hajnoczi
On Tue, May 02, 2023 at 05:19:46PM +0200, Kevin Wolf wrote: > Am 25.04.2023 um 19:26 hat Stefan Hajnoczi geschrieben: > > Only report a transport reset event to the guest after the SCSIDevice > > has been unrealized by qdev_simple_device_unplug_cb(). > > > > qdev_sim

Re: [PATCH v4 04/20] virtio-scsi: stop using aio_disable_external() during unplug

2023-05-01 Thread Stefan Hajnoczi
On Fri, Apr 28, 2023 at 04:22:55PM +0200, Kevin Wolf wrote: > Am 25.04.2023 um 19:27 hat Stefan Hajnoczi geschrieben: > > This patch is part of an effort to remove the aio_disable_external() > > API because it does not fit in a multi-queue block layer world where > >

[PATCH v4 19/20] virtio: do not set is_external=true on host notifiers

2023-04-25 Thread Stefan Hajnoczi
Host notifiers can now use is_external=false since virtio-blk and virtio-scsi no longer rely on is_external=true for drained sections. Signed-off-by: Stefan Hajnoczi --- hw/virtio/virtio.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio

[PATCH v4 16/20] virtio: make it possible to detach host notifier from any thread

2023-04-25 Thread Stefan Hajnoczi
-by: Stefan Hajnoczi --- hw/block/dataplane/virtio-blk.c | 2 ++ hw/scsi/virtio-scsi-dataplane.c | 9 + 2 files changed, 11 insertions(+) diff --git a/hw/block/dataplane/virtio-blk.c b/hw/block/dataplane/virtio-blk.c index b28d81737e..bd7cc6e76b 100644 --- a/hw/block/dataplane/virtio-blk.c +++ b

[PATCH v4 20/20] aio: remove aio_disable_external() API

2023-04-25 Thread Stefan Hajnoczi
, is_external, io_read, io_poll, io_poll_ready; @@ - aio_set_event_notifier(ctx, notifier, is_external, io_read, io_poll, io_poll_ready) + aio_set_event_notifier(ctx, notifier, io_read, io_poll, io_poll_ready) Reviewed-by: Juan Quintela Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Stefan

[PATCH v4 14/20] block/export: don't require AioContext lock around blk_exp_ref/unref()

2023-04-25 Thread Stefan Hajnoczi
The FUSE export calls blk_exp_ref/unref() without the AioContext lock. Instead of fixing the FUSE export, adjust blk_exp_ref/unref() so they work without the AioContext lock. This way it's less error-prone. Suggested-by: Paolo Bonzini Signed-off-by: Stefan Hajnoczi --- include/block/export.h

[PATCH v4 17/20] virtio-blk: implement BlockDevOps->drained_begin()

2023-04-25 Thread Stefan Hajnoczi
attaching/detaching ioeventfds if the data plane is started/stopped during a drained section. This should be rare, but maybe the mirror block job can trigger it. Signed-off-by: Stefan Hajnoczi --- hw/block/dataplane/virtio-blk.c | 17 +-- hw/block/virtio-blk.c | 38

[PATCH v4 18/20] virtio-scsi: implement BlockDevOps->drained_begin()

2023-04-25 Thread Stefan Hajnoczi
sees one pair of .drained_begin/end() calls. After this commit, virtio-scsi no longer depends on hw/virtio's ioeventfd aio_set_event_notifier(is_external=true). This commit is a step towards removing the aio_disable_external() API. Signed-off-by: Stefan Hajnoczi --- include/hw/scsi/scsi.h

[PATCH v4 13/20] block/export: rewrite vduse-blk drain code

2023-04-25 Thread Stefan Hajnoczi
in .drained_end(). This eliminates the need for the aio_set_fd_handler(is_external=true) flag, which is being removed from QEMU. This is a long list but splitting it into individual commits would probably lead to git bisect failures - the changes are all related. Signed-off-by: Stefan Hajnoczi

[PATCH v4 15/20] block/fuse: do not set is_external=true on FUSE fd

2023-04-25 Thread Stefan Hajnoczi
This is part of ongoing work to remove the aio_disable_external() API. Use BlockDevOps .drained_begin/end/poll() instead of aio_set_fd_handler(is_external=true). As a side-effect the FUSE export now follows AioContext changes like the other export types. Signed-off-by: Stefan Hajnoczi

[PATCH v4 11/20] xen-block: implement BlockDevOps->drained_begin()

2023-04-25 Thread Stefan Hajnoczi
on the dataplane instance, so move the blk_set_dev_ops() call after xen_block_dataplane_create(). Signed-off-by: Stefan Hajnoczi --- hw/block/dataplane/xen-block.h | 2 ++ hw/block/dataplane/xen-block.c | 42 +- hw/block/xen-block.c | 24

[PATCH v4 10/20] block: drain from main loop thread in bdrv_co_yield_to_drain()

2023-04-25 Thread Stefan Hajnoczi
. Signed-off-by: Stefan Hajnoczi --- include/sysemu/block-backend-common.h | 25 + block/io.c| 3 ++- 2 files changed, 15 insertions(+), 13 deletions(-) diff --git a/include/sysemu/block-backend-common.h b/include/sysemu/block-backend-common.h index

[PATCH v4 12/20] hw/xen: do not set is_external=true on evtchn fds

2023-04-25 Thread Stefan Hajnoczi
BlockDevOps .drained_begin/end() callbacks. It no longer relies on is_external=true so it is safe to pass is_external=false. This is part of ongoing work to remove the aio_disable_external() API. Signed-off-by: Stefan Hajnoczi --- hw/xen/xen-bus.c | 8 1 file changed, 4 insertions(+), 4

[PATCH v4 09/20] block: add blk_in_drain() API

2023-04-25 Thread Stefan Hajnoczi
The BlockBackend quiesce_counter is greater than zero during drained sections. Add an API to check whether the BlockBackend is in a drained section. The next patch will use this API. Signed-off-by: Stefan Hajnoczi --- include/sysemu/block-backend-global-state.h | 1 + block/block-backend.c

[PATCH v4 08/20] hw/xen: do not use aio_set_fd_handler(is_external=true) in xen_xenstore

2023-04-25 Thread Stefan Hajnoczi
-by: Stefan Hajnoczi --- hw/i386/kvm/xen_xenstore.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/i386/kvm/xen_xenstore.c b/hw/i386/kvm/xen_xenstore.c index 900679af8a..6e81bc8791 100644 --- a/hw/i386/kvm/xen_xenstore.c +++ b/hw/i386/kvm/xen_xenstore.c @@ -133,7 +133,7

[PATCH v4 06/20] block/export: wait for vhost-user-blk requests when draining

2023-04-25 Thread Stefan Hajnoczi
there are requests in flight. The in-flight counter needs to be atomic. Signed-off-by: Stefan Hajnoczi --- include/qemu/vhost-user-server.h | 4 +++- block/export/vhost-user-blk-server.c | 16 util/vhost-user-server.c | 14 ++ 3 files changed, 29

<    1   2   3   4   >