Re: [PATCH 00/18] block: Introduce a block graph rwlock

2022-12-07 Thread Kevin Wolf
Am 07.12.2022 um 15:12 hat Emanuele Giuseppe Esposito geschrieben: > > Emanuele Giuseppe Esposito (7): > > graph-lock: Implement guard macros > > async: Register/unregister aiocontext in graph lock list > > block: wrlock in bdrv_replace_child_noperm > > block: remove unnecessary

Re: [PATCH 00/18] block: Introduce a block graph rwlock

2022-12-07 Thread Emanuele Giuseppe Esposito
Am 07/12/2022 um 14:18 schrieb Kevin Wolf: > This series supersedes the first half of Emanuele's "Protect the block > layer with a rwlock: part 1". It introduces the basic infrastructure for > protecting the block graph (specifically parent/child links) with a > rwlock. Actually taking the

Re: [PATCH 1/1] qemu-iotests/stream-under-throttle: do not shutdown QEMU

2022-12-07 Thread Christian Borntraeger
Am 07.12.22 um 14:23 schrieb Thomas Huth: On 07/12/2022 14.14, Christian Borntraeger wrote: Without a kernel or boot disk a QEMU on s390 will exit (usually with a disabled wait state). This breaks the stream-under-throttle test case. Do not exit qemu if on s390. Signed-off-by: Christian

[PATCH 15/18] Mark assert_bdrv_graph_readable/writable() GRAPH_RD/WRLOCK

2022-12-07 Thread Kevin Wolf
Signed-off-by: Kevin Wolf --- include/block/block_int-common.h | 4 ++-- include/block/graph-lock.h | 4 ++-- block.c | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h index

[PATCH] block/mirror: add 'write-blocking-after-ready' copy mode

2022-12-07 Thread Fiona Ebner
The new copy mode starts out in 'background' mode and switches to 'write-blocking' mode once the job transitions to ready. Before switching to active mode and indicating that the drives are actively synced, it is necessary to have seen and handled all guest I/O. This is done by checking the dirty

Re: [PATCH 1/1] qemu-iotests/stream-under-throttle: do not shutdown QEMU

2022-12-07 Thread Thomas Huth
On 07/12/2022 14.14, Christian Borntraeger wrote: Without a kernel or boot disk a QEMU on s390 will exit (usually with a disabled wait state). This breaks the stream-under-throttle test case. Do not exit qemu if on s390. Signed-off-by: Christian Borntraeger ---

[PATCH 11/18] block: wrlock in bdrv_replace_child_noperm

2022-12-07 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Protect the main function where graph is modified. Signed-off-by: Emanuele Giuseppe Esposito Signed-off-by: Kevin Wolf --- block.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/block.c b/block.c index 44d59362d6..df52c6b012 100644

[PATCH 16/18] block-coroutine-wrapper.py: introduce annotations that take the graph rdlock

2022-12-07 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Add co_wrapper_bdrv_rdlock and co_wrapper_mixed_bdrv_rdlock option to the block-coroutine-wrapper.py script. This "_bdrv_rdlock" option takes and releases the graph rdlock when a coroutine function is created. This means that when used together with "_mixed",

[PATCH 12/18] block: remove unnecessary assert_bdrv_graph_writable()

2022-12-07 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito We don't protect bdrv->aio_context with the graph rwlock, so these assertions are not needed Signed-off-by: Emanuele Giuseppe Esposito Signed-off-by: Kevin Wolf --- block.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/block.c b/block.c index

[PATCH 06/18] clang-tsa: Add TSA_ASSERT() macro

2022-12-07 Thread Kevin Wolf
Signed-off-by: Kevin Wolf --- include/qemu/clang-tsa.h | 9 + 1 file changed, 9 insertions(+) diff --git a/include/qemu/clang-tsa.h b/include/qemu/clang-tsa.h index 0a3361dfc8..211ee0ae73 100644 --- a/include/qemu/clang-tsa.h +++ b/include/qemu/clang-tsa.h @@ -98,4 +98,13 @@ */

[PATCH 13/18] block: assert that graph read and writes are performed correctly

2022-12-07 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Remove the old assert_bdrv_graph_writable, and replace it with the new version using graph-lock API. See the function documentation for more information. Signed-off-by: Emanuele Giuseppe Esposito Signed-off-by: Kevin Wolf ---

[PATCH 18/18] block: GRAPH_RDLOCK for functions only called by co_wrappers

2022-12-07 Thread Kevin Wolf
The generated coroutine wrappers already take care to take the lock in the non-coroutine path, and assume that the lock is already taken in the coroutine path. The only thing we need to do for the wrapped function is adding the GRAPH_RDLOCK annotation. Doing so also allows us to mark the

[PATCH 02/18] graph-lock: Introduce a lock to protect block graph operations

2022-12-07 Thread Kevin Wolf
From: Paolo Bonzini Block layer graph operations are always run under BQL in the main loop. This is proved by the assertion qemu_in_main_thread() and its wrapper macro GLOBAL_STATE_CODE. However, there are also concurrent coroutines running in other iothreads that always try to traverse the

Re: [PATCH v15 1/6] qmp: add QMP command x-query-virtio

2022-12-07 Thread Markus Armbruster
Jonah Palmer writes: > On 12/2/22 10:21, Markus Armbruster wrote: >> Philippe Mathieu-Daudé writes: >> >>> On 2/12/22 13:23, Jonah Palmer wrote: On 11/30/22 11:16, Philippe Mathieu-Daudé wrote: > Hi, > > On 11/8/22 14:24, Jonah Palmer wrote: >> From: Laurent Vivier >>

[PATCH 01/18] block: Factor out bdrv_drain_all_begin_nopoll()

2022-12-07 Thread Kevin Wolf
Provide a separate function that just quiesces the users of a node to prevent new requests from coming in, but without waiting for the already in-flight I/O to complete. This function can be used in contexts where polling is not allowed. Signed-off-by: Kevin Wolf ---

[PATCH 04/18] async: Register/unregister aiocontext in graph lock list

2022-12-07 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Add/remove the AioContext in aio_context_list in graph-lock.c when it is created/destroyed. This allows using the graph locking operations from this AioContext. In order to allow linking util/async.c with binaries that don't include the block layer, introduce

[PATCH 14/18] graph-lock: TSA annotations for lock/unlock functions

2022-12-07 Thread Kevin Wolf
Signed-off-by: Kevin Wolf --- include/block/graph-lock.h | 80 +- block/graph-lock.c | 3 ++ 2 files changed, 73 insertions(+), 10 deletions(-) diff --git a/include/block/graph-lock.h b/include/block/graph-lock.h index 85e8a53b73..50b7e7b1b6 100644

[PATCH 07/18] clang-tsa: Add macros for shared locks

2022-12-07 Thread Kevin Wolf
Signed-off-by: Kevin Wolf --- include/qemu/clang-tsa.h | 4 1 file changed, 4 insertions(+) diff --git a/include/qemu/clang-tsa.h b/include/qemu/clang-tsa.h index 211ee0ae73..ba06fb8c92 100644 --- a/include/qemu/clang-tsa.h +++ b/include/qemu/clang-tsa.h @@ -62,6 +62,7 @@ * void

[PATCH 17/18] block: use co_wrapper_mixed_bdrv_rdlock in functions taking the rdlock

2022-12-07 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Take the rdlock already, before we add the assertions. All these functions either read the graph recursively, or call BlockDriver callbacks that will eventually need to be protected by the graph rdlock. Do it now to all functions together, because many of these

[PATCH 08/18] configure: Enable -Wthread-safety if present

2022-12-07 Thread Kevin Wolf
This enables clang's thread safety analysis (TSA), which we'll use to statically check the block graph locking. Signed-off-by: Kevin Wolf --- configure | 1 + 1 file changed, 1 insertion(+) diff --git a/configure b/configure index 26c7bc5154..45d693a241 100755 --- a/configure +++ b/configure

[PATCH 05/18] Import clang-tsa.h

2022-12-07 Thread Kevin Wolf
This defines macros that allow clang to perform Thread Safety Analysis based on function and variable annotations that specify the locking rules. On non-clang compilers, the annotations are ignored. Imported tsa.h from the original repository with the pthread_mutex_t wrapper removed:

[PATCH 09/18] test-bdrv-drain: Fix incorrrect drain assumptions

2022-12-07 Thread Kevin Wolf
The test case assumes that a drain only happens in one specific place where it drains explicitly. This assumption happened to hold true until now, but block layer functions may drain interally (any graph modifications are going to do that through bdrv_graph_wrlock()), so this is incorrect. Make

[PATCH 10/18] block: Fix locking in external_snapshot_prepare()

2022-12-07 Thread Kevin Wolf
bdrv_img_create() polls internally (when calling bdrv_create(), which is a co_wrapper), so it can't be called while holding the lock of any AioContext except the current one without causing deadlocks. Drop the lock around the call in external_snapshot_prepare(). Signed-off-by: Kevin Wolf ---

[PATCH 03/18] graph-lock: Implement guard macros

2022-12-07 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito Similar to the implementation in lockable.h, implement macros to automatically take and release the rdlock. Create the empty GraphLockable and GraphLockableMainloop structs only to use it as a type for G_DEFINE_AUTOPTR_CLEANUP_FUNC. Signed-off-by: Emanuele

[PATCH 00/18] block: Introduce a block graph rwlock

2022-12-07 Thread Kevin Wolf
This series supersedes the first half of Emanuele's "Protect the block layer with a rwlock: part 1". It introduces the basic infrastructure for protecting the block graph (specifically parent/child links) with a rwlock. Actually taking the reader lock in all necessary places is left for future

[PATCH 1/1] qemu-iotests/stream-under-throttle: do not shutdown QEMU

2022-12-07 Thread Christian Borntraeger
Without a kernel or boot disk a QEMU on s390 will exit (usually with a disabled wait state). This breaks the stream-under-throttle test case. Do not exit qemu if on s390. Signed-off-by: Christian Borntraeger --- tests/qemu-iotests/tests/stream-under-throttle | 2 ++ 1 file changed, 2

Re: [PATCH v2] tests/stream-under-throttle: New test

2022-12-07 Thread Christian Borntraeger
Am 07.12.22 um 13:56 schrieb Christian Borntraeger: Am 07.12.22 um 11:31 schrieb Christian Borntraeger: Am 14.11.22 um 10:52 schrieb Hanna Reitz: Test streaming a base image into the top image underneath two throttle nodes.  This was reported to make qemu 7.1 hang

Re: [PATCH v2] tests/stream-under-throttle: New test

2022-12-07 Thread Christian Borntraeger
Am 07.12.22 um 11:31 schrieb Christian Borntraeger: Am 14.11.22 um 10:52 schrieb Hanna Reitz: Test streaming a base image into the top image underneath two throttle nodes.  This was reported to make qemu 7.1 hang (https://gitlab.com/qemu-project/qemu/-/issues/1215), so this serves as a

Re: [PATCH v2] tests/stream-under-throttle: New test

2022-12-07 Thread Christian Borntraeger
Am 14.11.22 um 10:52 schrieb Hanna Reitz: Test streaming a base image into the top image underneath two throttle nodes. This was reported to make qemu 7.1 hang (https://gitlab.com/qemu-project/qemu/-/issues/1215), so this serves as a regression test. Signed-off-by: Hanna Reitz --- Based-on:

Re: [PATCH v15 1/6] qmp: add QMP command x-query-virtio

2022-12-07 Thread Jonah Palmer
On 12/2/22 10:21, Markus Armbruster wrote: Philippe Mathieu-Daudé writes: On 2/12/22 13:23, Jonah Palmer wrote: On 11/30/22 11:16, Philippe Mathieu-Daudé wrote: Hi, On 11/8/22 14:24, Jonah Palmer wrote: From: Laurent Vivier This new command lists all the instances of VirtIODevices with

Re: [PATCH] blockdev: add 'media=cdrom' argument to support usb cdrom emulated as cdrom

2022-12-07 Thread Paolo Bonzini
It should be like this: -device usb-bot,id=bot0 -device scsi-{cd,hd},bus=bot0.0,drive=drive0 Libvirt has the code to generate the options for SCSI controllers, but usb-bot only allows one disk attached to it so it's easier to make it a element. Paolo Il sab 3 dic 2022, 13:52 Zhipeng Lu ha