Re: [PATCH 2/6] qemu/main-loop: rename QEMU_IOTHREAD_LOCK_GUARD to QEMU_BQL_LOCK_GUARD

2023-11-30 Thread Harsh Prateek Bora
On 12/1/23 01:57, Stefan Hajnoczi wrote: On Thu, Nov 30, 2023 at 10:14:47AM +0100, Ilya Leoshkevich wrote: On Wed, 2023-11-29 at 16:26 -0500, Stefan Hajnoczi wrote: The name "iothread" is overloaded. Use the term Big QEMU Lock (BQL) instead, it is already widely used and unambiguous.

Re: [PATCH 1/6] system/cpus: rename qemu_mutex_lock_iothread() to qemu_bql_lock()

2023-11-30 Thread Harsh Prateek Bora
Hi Stefan, On 11/30/23 02:56, Stefan Hajnoczi wrote: The Big QEMU Lock (BQL) has many names and they are confusing. The actual QemuMutex variable is called qemu_global_mutex but it's commonly referred to as the BQL in discussions and some code comments. The locking APIs, however, are called

Re: [PATCH 10/12] scsi: remove outdated AioContext lock comment

2023-11-30 Thread Eric Blake
On Wed, Nov 29, 2023 at 02:55:51PM -0500, Stefan Hajnoczi wrote: > The SCSI subsystem no longer uses the AioContext lock. Request > processing runs exclusively in the BlockBackend's AioContext since > "scsi: only access SCSIDevice->requests from one thread" and hence the > lock is unnecessary. >

Re: [PATCH 09/12] docs: remove AioContext lock from IOThread docs

2023-11-30 Thread Eric Blake
On Wed, Nov 29, 2023 at 02:55:50PM -0500, Stefan Hajnoczi wrote: > Encourage the use of locking primitives and stop mentioning the > AioContext lock since it is being removed. > > Signed-off-by: Stefan Hajnoczi > --- > docs/devel/multiple-iothreads.txt | 45 +++ > 1

Re: [PATCH 1/6] system/cpus: rename qemu_mutex_lock_iothread() to qemu_bql_lock()

2023-11-30 Thread Eric Farman
On Wed, 2023-11-29 at 16:26 -0500, Stefan Hajnoczi wrote: > The Big QEMU Lock (BQL) has many names and they are confusing. The > actual QemuMutex variable is called qemu_global_mutex but it's > commonly > referred to as the BQL in discussions and some code comments. The > locking APIs, however,

Re: [PATCH 08/12] aio: remove aio_context_acquire()/aio_context_release() API

2023-11-30 Thread Eric Blake
On Wed, Nov 29, 2023 at 02:55:49PM -0500, Stefan Hajnoczi wrote: > Delete these functions because nothing calls these functions anymore. > > I introduced these APIs in commit 98563fc3ec44 ("aio: add > aio_context_acquire() and aio_context_release()") in 2014. It's with a > sigh of relief that I

Re: [PATCH 1/6] system/cpus: rename qemu_mutex_lock_iothread() to qemu_bql_lock()

2023-11-30 Thread BALATON Zoltan
On Thu, 30 Nov 2023, Stefan Hajnoczi wrote: On Thu, Nov 30, 2023 at 03:08:49PM -0500, Peter Xu wrote: On Wed, Nov 29, 2023 at 04:26:20PM -0500, Stefan Hajnoczi wrote: The Big QEMU Lock (BQL) has many names and they are confusing. The actual QemuMutex variable is called qemu_global_mutex but

Re: [PATCH 07/12] aio-wait: draw equivalence between AIO_WAIT_WHILE() and AIO_WAIT_WHILE_UNLOCKED()

2023-11-30 Thread Eric Blake
On Wed, Nov 29, 2023 at 02:55:48PM -0500, Stefan Hajnoczi wrote: > Now that the AioContext lock no longer exists, AIO_WAIT_WHILE() and > AIO_WAIT_WHILE_UNLOCKED() are equivalent. > > A future patch will get rid of AIO_WAIT_WHILE_UNLOCKED(). > > Signed-off-by: Stefan Hajnoczi > --- >

Re: [PATCH 06/12] scsi: remove AioContext locking

2023-11-30 Thread Eric Blake
On Wed, Nov 29, 2023 at 02:55:47PM -0500, Stefan Hajnoczi wrote: > The AioContext lock no longer has any effect. Remove it. > > Signed-off-by: Stefan Hajnoczi > --- > include/hw/virtio/virtio-scsi.h | 14 -- > hw/scsi/scsi-bus.c | 2 -- > hw/scsi/scsi-disk.c

Re: [PATCH 05/12] block: remove AioContext locking

2023-11-30 Thread Eric Blake
On Wed, Nov 29, 2023 at 02:55:46PM -0500, Stefan Hajnoczi wrote: > This is the big patch that removes > aio_context_acquire()/aio_context_release() from the block layer and > affected block layer users. > > There isn't a clean way to split this patch and the reviewers are likely > the same group

Re: [PATCH 1/6] system/cpus: rename qemu_mutex_lock_iothread() to qemu_bql_lock()

2023-11-30 Thread Peter Xu
On Thu, Nov 30, 2023 at 03:43:25PM -0500, Stefan Hajnoczi wrote: > On Thu, Nov 30, 2023 at 03:08:49PM -0500, Peter Xu wrote: > > On Wed, Nov 29, 2023 at 04:26:20PM -0500, Stefan Hajnoczi wrote: > > > The Big QEMU Lock (BQL) has many names and they are confusing. The > > > actual QemuMutex variable

Re: [PATCH 1/6] system/cpus: rename qemu_mutex_lock_iothread() to qemu_bql_lock()

2023-11-30 Thread Stefan Hajnoczi
On Thu, Nov 30, 2023 at 03:08:49PM -0500, Peter Xu wrote: > On Wed, Nov 29, 2023 at 04:26:20PM -0500, Stefan Hajnoczi wrote: > > The Big QEMU Lock (BQL) has many names and they are confusing. The > > actual QemuMutex variable is called qemu_global_mutex but it's commonly > > referred to as the BQL

Re: [PATCH 6/6] Rename "QEMU global mutex" to "BQL" in comments and docs

2023-11-30 Thread Stefan Hajnoczi
On Thu, Nov 30, 2023 at 02:49:48PM +0100, Philippe Mathieu-Daudé wrote: > On 29/11/23 22:26, Stefan Hajnoczi wrote: > > The term "QEMU global mutex" is identical to the more widely used Big > > QEMU Lock ("BQL"). Update the code comments and documentation to use > > "BQL" instead of "QEMU global

Re: [PATCH 5/6] Replace "iothread lock" with "BQL" in comments

2023-11-30 Thread Stefan Hajnoczi
On Thu, Nov 30, 2023 at 02:47:49PM +0100, Philippe Mathieu-Daudé wrote: > Hi Stefan, > > On 29/11/23 22:26, Stefan Hajnoczi wrote: > > The term "iothread lock" is obsolete. The APIs use Big QEMU Lock (BQL) > > in their names. Update the code comments to use "BQL" instead of > > "iothread lock". >

Re: [PATCH 4/6] system/cpus: rename qemu_global_mutex to qemu_bql

2023-11-30 Thread Stefan Hajnoczi
On Thu, Nov 30, 2023 at 02:44:07PM +0100, Philippe Mathieu-Daudé wrote: > Hi Stefan, > > On 29/11/23 22:26, Stefan Hajnoczi wrote: > > The APIs using qemu_global_mutex now follow the Big QEMU Lock (BQL) > > nomenclature. It's a little strange that the actual QemuMutex variable > > that embodies

Re: [PATCH 2/6] qemu/main-loop: rename QEMU_IOTHREAD_LOCK_GUARD to QEMU_BQL_LOCK_GUARD

2023-11-30 Thread Stefan Hajnoczi
On Thu, Nov 30, 2023 at 10:14:47AM +0100, Ilya Leoshkevich wrote: > On Wed, 2023-11-29 at 16:26 -0500, Stefan Hajnoczi wrote: > > The name "iothread" is overloaded. Use the term Big QEMU Lock (BQL) > > instead, it is already widely used and unambiguous. > > > > Signed-off-by: Stefan Hajnoczi > >

Re: [PATCH v2 1/2] block: commit: Allow users to request only format driver names in backing file format

2023-11-30 Thread Peter Krempa
On Thu, Nov 30, 2023 at 13:24:18 -0600, Eric Blake wrote: > On Thu, Nov 30, 2023 at 05:06:03PM +0100, Peter Krempa wrote: > > Introduce a new flag 'backing_file_format_no_protocol' for the > > block-commit QMP command which instructs the internals to use 'raw' > > instead of the protocol driver in

Re: [PATCH 1/6] system/cpus: rename qemu_mutex_lock_iothread() to qemu_bql_lock()

2023-11-30 Thread Peter Xu
On Wed, Nov 29, 2023 at 04:26:20PM -0500, Stefan Hajnoczi wrote: > The Big QEMU Lock (BQL) has many names and they are confusing. The > actual QemuMutex variable is called qemu_global_mutex but it's commonly > referred to as the BQL in discussions and some code comments. The > locking APIs,

Re: [PATCH v2 1/2] block: commit: Allow users to request only format driver names in backing file format

2023-11-30 Thread Eric Blake
On Thu, Nov 30, 2023 at 05:06:03PM +0100, Peter Krempa wrote: > Introduce a new flag 'backing_file_format_no_protocol' for the > block-commit QMP command which instructs the internals to use 'raw' > instead of the protocol driver in case when a image is used without a > dummy 'raw' wrapper. > >

Re: [PATCH v2 2/2] block: stream: Allow users to request only format driver names in backing file format

2023-11-30 Thread Eric Blake
On Thu, Nov 30, 2023 at 05:06:04PM +0100, Peter Krempa wrote: > Introduce a new flag 'backing_file_format_no_protocol' for the > block-commit QMP command which instructs the internals to use 'raw' > instead of the protocol driver in case when a image is used without a > dummy 'raw' wrapper. Same

Re: [PATCH 04/12] graph-lock: remove AioContext locking

2023-11-30 Thread Eric Blake
On Wed, Nov 29, 2023 at 02:55:45PM -0500, Stefan Hajnoczi wrote: > Stop acquiring/releasing the AioContext lock in > bdrv_graph_wrlock()/bdrv_graph_unlock() since the lock no longer has any > effect. > > The distinction between bdrv_graph_wrunlock() and > bdrv_graph_wrunlock_ctx() becomes

Re: [PATCH 03/12] aio: make aio_context_acquire()/aio_context_release() a no-op

2023-11-30 Thread Eric Blake
On Wed, Nov 29, 2023 at 02:55:44PM -0500, Stefan Hajnoczi wrote: > aio_context_acquire()/aio_context_release() has been replaced by > fine-grained locking to protect state shared by multiple threads. The > AioContext lock still plays the role of balancing locking in > AIO_WAIT_WHILE() and many

[PATCH v2 1/2] block: commit: Allow users to request only format driver names in backing file format

2023-11-30 Thread Peter Krempa
Introduce a new flag 'backing_file_format_no_protocol' for the block-commit QMP command which instructs the internals to use 'raw' instead of the protocol driver in case when a image is used without a dummy 'raw' wrapper. The flag is designed such that it can be always asserted by management

[PATCH v2 2/2] block: stream: Allow users to request only format driver names in backing file format

2023-11-30 Thread Peter Krempa
Introduce a new flag 'backing_file_format_no_protocol' for the block-commit QMP command which instructs the internals to use 'raw' instead of the protocol driver in case when a image is used without a dummy 'raw' wrapper. The flag is designed such that it can be always asserted by management

[PATCH v2 0/2] block: commit/stream: Allow users to request only format driver names in backing file format

2023-11-30 Thread Peter Krempa
Please see patches for rationale. Libvirt patches using this new flag will be posted soon-ish (after cleanup). v2: - fixed mistaken argument order in 'hmp_block_stream' - changed version in docs to 9.0 as getting this into RC 3 probably isn't realistic Peter Krempa (2): block: commit:

Re: [PATCH 02/12] tests: remove aio_context_acquire() tests

2023-11-30 Thread Eric Blake
On Wed, Nov 29, 2023 at 02:55:43PM -0500, Stefan Hajnoczi wrote: > The aio_context_acquire() API is being removed. Drop the test case that > calls the API. > > Signed-off-by: Stefan Hajnoczi > --- > tests/unit/test-aio.c | 67 +-- > 1 file changed, 1

Re: [PATCH 01/12] virtio-scsi: replace AioContext lock with tmf_bh_lock

2023-11-30 Thread Eric Blake
On Wed, Nov 29, 2023 at 02:55:42PM -0500, Stefan Hajnoczi wrote: > Protect the Task Management Function BH state with a lock. The TMF BH > runs in the main loop thread. An IOThread might process a TMF at the > same time as the TMF BH is running. Therefore tmf_bh_list and tmf_bh > must be protected

Re: [PATCH 6/6] Rename "QEMU global mutex" to "BQL" in comments and docs

2023-11-30 Thread Philippe Mathieu-Daudé
On 29/11/23 22:26, Stefan Hajnoczi wrote: The term "QEMU global mutex" is identical to the more widely used Big QEMU Lock ("BQL"). Update the code comments and documentation to use "BQL" instead of "QEMU global mutex". Signed-off-by: Stefan Hajnoczi --- docs/devel/multi-thread-tcg.rst | 7

Re: [PATCH 5/6] Replace "iothread lock" with "BQL" in comments

2023-11-30 Thread Philippe Mathieu-Daudé
Hi Stefan, On 29/11/23 22:26, Stefan Hajnoczi wrote: The term "iothread lock" is obsolete. The APIs use Big QEMU Lock (BQL) in their names. Update the code comments to use "BQL" instead of "iothread lock". Signed-off-by: Stefan Hajnoczi --- docs/devel/reset.rst | 2 +-

Re: [PATCH 4/6] system/cpus: rename qemu_global_mutex to qemu_bql

2023-11-30 Thread Philippe Mathieu-Daudé
Hi Stefan, On 29/11/23 22:26, Stefan Hajnoczi wrote: The APIs using qemu_global_mutex now follow the Big QEMU Lock (BQL) nomenclature. It's a little strange that the actual QemuMutex variable that embodies the BQL is called qemu_global_mutex instead of qemu_bql. Rename it for consistency.

Re: [PATCH 3/6] qemu/main-loop: rename qemu_cond_wait_iothread() to qemu_cond_wait_bql()

2023-11-30 Thread Philippe Mathieu-Daudé
On 29/11/23 22:26, Stefan Hajnoczi wrote: The name "iothread" is overloaded. Use the term Big QEMU Lock (BQL) instead, it is already widely used and unambiguous. Signed-off-by: Stefan Hajnoczi --- include/qemu/main-loop.h | 8 accel/tcg/tcg-accel-ops-rr.c | 4 ++--

Re: [PATCH 4/6] system/cpus: rename qemu_global_mutex to qemu_bql

2023-11-30 Thread Cédric Le Goater
On 11/29/23 22:26, Stefan Hajnoczi wrote: The APIs using qemu_global_mutex now follow the Big QEMU Lock (BQL) nomenclature. It's a little strange that the actual QemuMutex variable that embodies the BQL is called qemu_global_mutex instead of qemu_bql. Rename it for consistency. Signed-off-by:

Re: [PATCH 3/6] qemu/main-loop: rename qemu_cond_wait_iothread() to qemu_cond_wait_bql()

2023-11-30 Thread Cédric Le Goater
On 11/29/23 22:26, Stefan Hajnoczi wrote: The name "iothread" is overloaded. Use the term Big QEMU Lock (BQL) instead, it is already widely used and unambiguous. Signed-off-by: Stefan Hajnoczi Reviewed-by: Cédric Le Goater Thanks, C.

Re: [PATCH 2/6] qemu/main-loop: rename QEMU_IOTHREAD_LOCK_GUARD to QEMU_BQL_LOCK_GUARD

2023-11-30 Thread Cédric Le Goater
On 11/29/23 22:26, Stefan Hajnoczi wrote: The name "iothread" is overloaded. Use the term Big QEMU Lock (BQL) instead, it is already widely used and unambiguous. Signed-off-by: Stefan Hajnoczi Reviewed-by: Cédric Le Goater Thanks, C.

Re: [PATCH 1/6] system/cpus: rename qemu_mutex_lock_iothread() to qemu_bql_lock()

2023-11-30 Thread Cédric Le Goater
On 11/29/23 22:26, Stefan Hajnoczi wrote: The Big QEMU Lock (BQL) has many names and they are confusing. The actual QemuMutex variable is called qemu_global_mutex but it's commonly referred to as the BQL in discussions and some code comments. The locking APIs, however, are called

Re: [PATCH 2/6] qemu/main-loop: rename QEMU_IOTHREAD_LOCK_GUARD to QEMU_BQL_LOCK_GUARD

2023-11-30 Thread Ilya Leoshkevich
On Wed, 2023-11-29 at 16:26 -0500, Stefan Hajnoczi wrote: > The name "iothread" is overloaded. Use the term Big QEMU Lock (BQL) > instead, it is already widely used and unambiguous. > > Signed-off-by: Stefan Hajnoczi > --- >  include/qemu/main-loop.h  | 20 ++-- >  

Re: [PATCH 2/6] qemu/main-loop: rename QEMU_IOTHREAD_LOCK_GUARD to QEMU_BQL_LOCK_GUARD

2023-11-30 Thread David Woodhouse
On Wed, 2023-11-29 at 16:26 -0500, Stefan Hajnoczi wrote: > The name "iothread" is overloaded. Use the term Big QEMU Lock (BQL) > instead, it is already widely used and unambiguous. > > Signed-off-by: Stefan Hajnoczi Acked-by: David Woodhouse smime.p7s Description: S/MIME cryptographic

Re: [PATCH 1/6] system/cpus: rename qemu_mutex_lock_iothread() to qemu_bql_lock()

2023-11-30 Thread David Woodhouse
On Wed, 2023-11-29 at 16:26 -0500, Stefan Hajnoczi wrote: > The Big QEMU Lock (BQL) has many names and they are confusing. The > actual QemuMutex variable is called qemu_global_mutex but it's commonly > referred to as the BQL in discussions and some code comments. The > locking APIs, however, are

Re: [PATCH 1/6] system/cpus: rename qemu_mutex_lock_iothread() to qemu_bql_lock()

2023-11-30 Thread Fabiano Rosas
Stefan Hajnoczi writes: > The Big QEMU Lock (BQL) has many names and they are confusing. The > actual QemuMutex variable is called qemu_global_mutex but it's commonly > referred to as the BQL in discussions and some code comments. The > locking APIs, however, are called

Re: [RFC v2 0/7] Add persistence to NVMe ZNS emulation

2023-11-30 Thread Sam Li
Markus Armbruster 于2023年11月30日周四 18:11写道: > > Sam Li writes: > > > ZNS emulation follows NVMe ZNS spec but the state of namespace > > zones does not persist accross restarts of QEMU. This patch makes the > > metadata of ZNS emulation persistent by using new block layer APIs and > > the qcow2 img

Re: [RFC v2 2/7] qcow2: add zd_extension configurations to zoned metadata

2023-11-30 Thread Markus Armbruster
Sam Li writes: > Zone descriptor data is host definied data that is associated with > each zone. Add zone descriptor extensions to zonedmeta struct. > > Signed-off-by: Sam Li [...] > struct BlockBackendRootState { > diff --git a/qapi/block-core.json b/qapi/block-core.json > index

Re: [RFC v2 0/7] Add persistence to NVMe ZNS emulation

2023-11-30 Thread Markus Armbruster
Sam Li writes: > ZNS emulation follows NVMe ZNS spec but the state of namespace > zones does not persist accross restarts of QEMU. This patch makes the > metadata of ZNS emulation persistent by using new block layer APIs and > the qcow2 img as backing file. It is the second part after the

[PATCH] blockcommit: Reopen base image as RO after abort

2023-11-30 Thread Alexander Ivanov
If a blockcommit is aborted the base image remains in RW mode, that leads to a fail of subsequent live migration. How to reproduce: $ virsh snapshot-create-as vm snp1 --disk-only *** write something to the disk *** $ virsh blockcommit vm vda --active --shallow && virsh blockjob vm vda

Re: [RFC v2 1/7] docs/qcow2: add zd_extension_size option to the zoned format feature

2023-11-30 Thread Markus Armbruster
Sam Li writes: > Signed-off-by: Sam Li > --- > docs/interop/qcow2.txt | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt > index 0f1938f056..458d05371a 100644 > --- a/docs/interop/qcow2.txt > +++ b/docs/interop/qcow2.txt > @@ -428,6

Re: [PATCH 05/12] block: remove AioContext locking

2023-11-30 Thread Paul Durrant
On 29/11/2023 19:55, Stefan Hajnoczi wrote: This is the big patch that removes aio_context_acquire()/aio_context_release() from the block layer and affected block layer users. There isn't a clean way to split this patch and the reviewers are likely the same group of people, so I decided to do

Re: [PATCH 2/6] qemu/main-loop: rename QEMU_IOTHREAD_LOCK_GUARD to QEMU_BQL_LOCK_GUARD

2023-11-30 Thread Paul Durrant
On 29/11/2023 21:26, Stefan Hajnoczi wrote: The name "iothread" is overloaded. Use the term Big QEMU Lock (BQL) instead, it is already widely used and unambiguous. Signed-off-by: Stefan Hajnoczi --- include/qemu/main-loop.h | 20 ++-- hw/i386/kvm/xen_evtchn.c | 14

Re: [PATCH 1/6] system/cpus: rename qemu_mutex_lock_iothread() to qemu_bql_lock()

2023-11-30 Thread Paul Durrant
On 29/11/2023 21:26, Stefan Hajnoczi wrote: The Big QEMU Lock (BQL) has many names and they are confusing. The actual QemuMutex variable is called qemu_global_mutex but it's commonly referred to as the BQL in discussions and some code comments. The locking APIs, however, are called

Re: [PATCH v6 2/4] qcow2: add configurations for zoned format extension

2023-11-30 Thread Markus Armbruster
Sam Li writes: > To configure the zoned format feature on the qcow2 driver, it > requires settings as: the device size, zone model, zone size, > zone capacity, number of conventional zones, limits on zone > resources (max append bytes, max open zones, and max_active_zones). > > To create a qcow2

Re: [PATCH v6 0/4] Add full zoned storage emulation to qcow2 driver

2023-11-30 Thread Markus Armbruster
Clang reports ../block/qcow2.c:4066:5: error: mutex 'graph_lock' is not held on every path through here [-Werror,-Wthread-safety-analysis] blk_co_unref(blk); ^ ../block/qcow2.c:3928:5: note: mutex acquired here bdrv_graph_co_rdlock(); ^ ../block/qcow2.c:4066:5: error: mutex