[RFC PATCH 2/6] introduce BDRV_POLL_WHILE_UNLOCKED

2021-12-13 Thread Emanuele Giuseppe Esposito
Same as BDRV_POLL_WHILE, but uses AIO_WAIT_WHILE_UNLOCKED. Signed-off-by: Emanuele Giuseppe Esposito --- include/block/block-global-state.h | 5 + 1 file changed, 5 insertions(+) diff --git a/include/block/block-global-state.h b/include/block/block-global-state.h index 7a6b065101

[PATCH v2 1/4] block_int: make bdrv_backing_overridden static

2021-11-30 Thread Emanuele Giuseppe Esposito
bdrv_backing_overridden is only used in block.c, so there is no need to leave it in block_int.h Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 4 +++- include/block/block_int.h | 3 --- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/block.c b/block.c

[PATCH v2 4/4] include/sysemu/blockdev.h: remove drive_get_max_devs

2021-11-30 Thread Emanuele Giuseppe Esposito
Remove drive_get_max_devs, as it is not used by anyone. Last use was removed in commit 8f2d75e81d5 ("hw: Drop superfluous special checks for orphaned -drive"). Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Philippe Mathieu-Daudé --- blockdev.c

[PATCH v2 2/4] include/sysemu/blockdev.c: introduce block_if_name

2021-11-30 Thread Emanuele Giuseppe Esposito
Add a getter function for the if_name array, so that also outside functions can access it. Signed-off-by: Emanuele Giuseppe Esposito --- blockdev.c| 5 + include/sysemu/blockdev.h | 1 + 2 files changed, 6 insertions(+) diff --git a/blockdev.c b/blockdev.c index b35072644e

[PATCH v2 3/4] include/sysemu/blockdev.h: move drive_add and inline drive_def

2021-11-30 Thread Emanuele Giuseppe Esposito
: Emanuele Giuseppe Esposito --- block/monitor/block-hmp-cmds.c | 2 +- blockdev.c | 25 - include/sysemu/blockdev.h | 4 softmmu/vl.c | 25 - 4 files changed, 25 insertions(+), 31 deletions(-) diff --git a

[PATCH v2 0/4] block: minor refactoring in preparation to the block layer API split

2021-11-30 Thread Emanuele Giuseppe Esposito
nctions in blockdev.h and block_int.h. Signed-off-by: Emanuele Giuseppe Esposito --- v2: * Apply Philippe comments, instead of renaming a make if_name public, create a getter method (discard old patch 2). Emanuele Giuseppe Esposito (4): block_int: make bdrv_backing_overridden static incl

Re: [PATCH v5 00/31] block layer: split block APIs in global state and I/O

2021-11-30 Thread Emanuele Giuseppe Esposito
On 29/11/2021 14:32, Stefan Hajnoczi wrote: On Wed, Nov 24, 2021 at 01:43:47AM -0500, Emanuele Giuseppe Esposito wrote: v5 -> v6: * In short, apply all Hanna's comments. More in details, the following functions in the following headers have been moved: block

[PATCH v5 28/31] block.c: assert BQL lock held in bdrv_co_invalidate_cache

2021-11-23 Thread Emanuele Giuseppe Esposito
ensuring that bs (and parents) .open_flags does not contain BDRV_O_INACTIVE. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 26 ++ 1 file changed, 26 insertions(+) diff --git a/block.c b/block.c index a0309f827d..805974676b 100644 --- a/block.c +++ b/block.c @@ -6574,6

[PATCH v5 27/31] job.h: assertions in the callers of JobDriver funcion pointers

2021-11-23 Thread Emanuele Giuseppe Esposito
Also assert that job->run() callback is called in the job aiocontext. Signed-off-by: Emanuele Giuseppe Esposito --- job.c | 10 ++ 1 file changed, 10 insertions(+) diff --git a/job.c b/job.c index dbfa67bb0a..bb57ec6887 100644 --- a/job.c +++ b/job.c @@ -380,6 +380,8 @@ void job_

[PATCH v5 25/31] block-backend-common.h: split function pointers in BlockDevOps

2021-11-23 Thread Emanuele Giuseppe Esposito
Assertions in the callers of the function pointrs are already added by previous patches. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi Reviewed-by: Philippe Mathieu-Daudé --- include/sysemu/block-backend-common.h | 28 ++- 1 file changed, 23

[PATCH v5 23/31] block_int-common.h: split function pointers in BdrvChildClass

2021-11-23 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito --- include/block/block_int-common.h | 67 +++- 1 file changed, 40 insertions(+), 27 deletions(-) diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h index 0e63dc694f..3ceb2365a8 100644 --- a

[PATCH v5 22/31] block_int-common.h: assertion in the callers of BlockDriver function pointers

2021-11-23 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- block.c| 18 ++ block/create.c | 10 ++ 2 files changed, 28 insertions(+) diff --git a/block.c b/block.c index b77ab0a104..180884b8c0 100644 --- a/block.c +++ b/block.c @@ -526,6 +526,7

[PATCH v5 18/31] include/block/snapshot: global state API + assertions

2021-11-23 Thread Emanuele Giuseppe Esposito
Snapshots run also under the BQL lock, so they all are in the global state API. The aiocontext lock that they hold is currently an overkill and in future could be removed. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Paolo Bonzini Reviewed-by: Stefan Hajnoczi --- include/block

[PATCH v5 30/31] crypto: delegate permission functions to JobDriver .pre_run

2021-11-23 Thread Emanuele Giuseppe Esposito
e job aiocontext. This has also the benefit of applying the same permission operation to all amend implementations, not only luks. Signed-off-by: Emanuele Giuseppe Esposito --- block/amend.c | 20 block/crypto.c | 18 -- 2 files changed, 32 insertions(

[PATCH v5 31/31] block.c: assertions to the block layer permissions API

2021-11-23 Thread Emanuele Giuseppe Esposito
Now that we "covered" the three main cases where the permission API was being used under BQL (fuse, amend and invalidate_cache), we can safely assert for the permission functions implemented in block.c Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 12 1 file c

[PATCH v5 21/31] block_int-common.h: split function pointers in BlockDriver

2021-11-23 Thread Emanuele Giuseppe Esposito
Similar to the header split, also the function pointers in BlockDriver can be split in I/O and global state. Signed-off-by: Emanuele Giuseppe Esposito --- include/block/block_int-common.h | 440 +-- 1 file changed, 235 insertions(+), 205 deletions(-) diff --git a

[PATCH v5 08/31] assertions for block_int global state API

2021-11-23 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 15 +++ block/backup.c | 1 + block/block-backend.c | 3 +++ block/commit.c | 2 ++ block/dirty-bitmap.c| 1 + block/io.c

[PATCH v5 26/31] job.h: split function pointers in JobDriver

2021-11-23 Thread Emanuele Giuseppe Esposito
The job API will be handled separately in another serie. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job.h | 22 ++ 1 file changed, 22 insertions(+) diff --git a/include/qemu/job.h b/include/qemu/job.h index 6e67b6977f..4ea7a4a0cd 100644 --- a/include/qemu

[PATCH v5 29/31] jobs: introduce pre_run function in JobDriver

2021-11-23 Thread Emanuele Giuseppe Esposito
.pre_run takes care of doing some initial job setup just before the job is run inside the coroutine. Doing so can be useful to invoke GS functions that require the BQL held. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job.h | 9 + job.c | 13 + 2

[PATCH v5 19/31] block/copy-before-write.h: global state API + assertions

2021-11-23 Thread Emanuele Giuseppe Esposito
copy-before-write functions always run under BQL lock. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Paolo Bonzini Reviewed-by: Stefan Hajnoczi --- block/copy-before-write.h | 7 +++ block/copy-before-write.c | 2 ++ 2 files changed, 9 insertions(+) diff --git a/block/copy

[PATCH v5 10/31] block.c: modify .attach and .detach callbacks of child_of_bds

2021-11-23 Thread Emanuele Giuseppe Esposito
he drain function so it will still be included in the operation, despite not being in the list. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/block.c b/block.c index 522a273140..5516c84ec4 100644 --- a/block.c +++ b

[PATCH v5 24/31] block_int-common.h: assertions in the callers of BdrvChildClass function pointers

2021-11-23 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 9 + 1 file changed, 9 insertions(+) diff --git a/block.c b/block.c index 180884b8c0..a0309f827d 100644 --- a/block.c +++ b/block.c @@ -1491,6 +1491,7 @@ const BdrvChildClass child_of_bds = { AioContext

[PATCH v5 20/31] block/coroutines: I/O API

2021-11-23 Thread Emanuele Giuseppe Esposito
block coroutines functions run in different aiocontext, and are not protected by the BQL. Therefore are I/O. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Paolo Bonzini Reviewed-by: Stefan Hajnoczi --- block/coroutines.h | 6 ++ 1 file changed, 6 insertions(+) diff --git a/block

[PATCH v5 09/31] block: introduce assert_bdrv_graph_writable

2021-11-23 Thread Emanuele Giuseppe Esposito
. Because adding drains requires additional discussions, they will be added in future series. Signed-off-by: Emanuele Giuseppe Esposito --- include/block/block_int-global-state.h | 10 +- block.c| 4 block/io.c | 11

[PATCH v5 16/31] include/sysemu/blockdev.h: global state API

2021-11-23 Thread Emanuele Giuseppe Esposito
blockdev functions run always under the BQL lock. Signed-off-by: Emanuele Giuseppe Esposito --- include/sysemu/blockdev.h | 13 ++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/include/sysemu/blockdev.h b/include/sysemu/blockdev.h index c4b7b8b54e..e53eb91be6 100644

[PATCH v5 13/31] block.c: add assertions to static functions

2021-11-23 Thread Emanuele Giuseppe Esposito
Following the assertion derived from the API split, propagate the assertion also in the static functions. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 45 - 1 file changed, 44 insertions(+), 1 deletion(-) diff --git a/block.c b/block.c

[PATCH v5 15/31] assertions for blockjob.h global state API

2021-11-23 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- blockjob.c | 10 ++ 1 file changed, 10 insertions(+) diff --git a/blockjob.c b/blockjob.c index 10c807413e..74476af473 100644 --- a/blockjob.c +++ b/blockjob.c @@ -62,6 +62,7 @@ static bool is_block_job(Job *job

[PATCH v5 17/31] assertions for blockdev.h global state API

2021-11-23 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Paolo Bonzini Reviewed-by: Stefan Hajnoczi --- block/block-backend.c | 3 +++ blockdev.c| 15 +++ 2 files changed, 18 insertions(+) diff --git a/block/block-backend.c b/block/block-backend.c index 56670a774f

[PATCH v5 07/31] include/block/block_int: split header into I/O and global state API

2021-11-23 Thread Emanuele Giuseppe Esposito
ff-by: Emanuele Giuseppe Esposito --- include/block/block_int-common.h | 1161 +++ include/block/block_int-global-state.h | 315 + include/block/block_int-io.h | 167 +++ include/block/block_int.h | 1475 +--- block

[PATCH v5 11/31] include/block/blockjob_int.h: split header into I/O and GS API

2021-11-23 Thread Emanuele Giuseppe Esposito
Since the I/O functions are not many, keep a single file. Also split the function pointers in BlockJobDriver. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- include/block/blockjob_int.h | 28 1 file changed, 28 insertions(+) diff --git

[PATCH v5 14/31] include/block/blockjob.h: global state API

2021-11-23 Thread Emanuele Giuseppe Esposito
blockjob functions run always under the BQL lock. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- include/block/blockjob.h | 9 + 1 file changed, 9 insertions(+) diff --git a/include/block/blockjob.h b/include/block/blockjob.h index d200f33c10..fa0c3f7a47

[PATCH v5 06/31] block/block-backend.c: assertions for block-backend

2021-11-23 Thread Emanuele Giuseppe Esposito
All the global state (GS) API functions will check that qemu_in_main_thread() returns true. If not, it means that the safety of BQL cannot be guaranteed, and they need to be moved to I/O. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c | 83

[PATCH v5 12/31] assertions for blockjob_int.h

2021-11-23 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- blockjob.c | 4 1 file changed, 4 insertions(+) diff --git a/blockjob.c b/blockjob.c index 4bad1408cb..10c807413e 100644 --- a/blockjob.c +++ b/blockjob.c @@ -84,6 +84,7 @@ BlockJob *block_job_get(const char *id

[PATCH v5 03/31] assertions for block global state API

2021-11-23 Thread Emanuele Giuseppe Esposito
All the global state (GS) API functions will check that qemu_in_main_thread() returns true. If not, it means that the safety of BQL cannot be guaranteed, and they need to be moved to I/O. Signed-off-by: Emanuele Giuseppe Esposito --- block.c| 135

[PATCH v5 05/31] block-backend: special comments for blk_set/get_perm due to fuse

2021-11-23 Thread Emanuele Giuseppe Esposito
code. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-backend.c | 10 ++ block/export/fuse.c | 16 ++-- 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/block/block-backend.c b/block/block-backend.c index 1f0bda578e..7a4b50a8f0 100644 --- a/block

[PATCH v5 02/31] include/block/block: split header into I/O and global state API

2021-11-23 Thread Emanuele Giuseppe Esposito
er view on what needs what kind of protection. block-common.h contains common structures shared by both headers. block.h is left there for legacy and to avoid changing all includes in all c files that use the block APIs. Assertions are added in the next patch. Signed-off-by: Emanuele Giusepp

[PATCH v5 00/31] block layer: split block APIs in global state and I/O

2021-11-23 Thread Emanuele Giuseppe Esposito
;block: minor refactoring in preparation to the block layer API split". Based-on: <20211124063640.3118897-1-eespo...@redhat.com> Signed-off-by: Emanuele Giuseppe Esposito --- v5 -> v6: * In short, apply all Hanna's comments. More in details, the following function

[PATCH 4/4] include/sysemu/blockdev.h: remove drive_get_max_devs

2021-11-23 Thread Emanuele Giuseppe Esposito
Remove drive_get_max_devs, as it is not used by anyone. Signed-off-by: Emanuele Giuseppe Esposito --- include/sysemu/blockdev.h | 1 - blockdev.c| 17 - 2 files changed, 18 deletions(-) diff --git a/include/sysemu/blockdev.h b/include/sysemu/blockdev.h index

[PATCH v5 04/31] include/sysemu/block-backend: split header into I/O and global state (GS) API

2021-11-23 Thread Emanuele Giuseppe Esposito
ared between the two headers, and the functions that can't be categorized as I/O or global state. Assertions are added in the next patch. Signed-off-by: Emanuele Giuseppe Esposito --- include/sysemu/block-backend-common.h | 84 ++ include/sysemu/block-backend-global-state.h | 121

[PATCH v5 01/31] main-loop.h: introduce qemu_in_main_thread()

2021-11-23 Thread Emanuele Giuseppe Esposito
next patches fail despite the AioContext is still the main loop. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- include/qemu/main-loop.h | 13 + softmmu/cpus.c | 5 + stubs/iothread-lock.c| 5 + 3 files changed, 23 insertions(+) diff

[PATCH 3/4] include/sysemu/blockdev.h: move drive_add and inline drive_def

2021-11-23 Thread Emanuele Giuseppe Esposito
: Emanuele Giuseppe Esposito --- include/sysemu/blockdev.h | 6 ++ block/monitor/block-hmp-cmds.c | 2 +- blockdev.c | 27 +-- softmmu/vl.c | 25 - 4 files changed, 28 insertions(+), 32 deletions(-) diff

[PATCH 1/4] block_int: make bdrv_backing_overridden static

2021-11-23 Thread Emanuele Giuseppe Esposito
bdrv_backing_overridden is only used in block.c, so there is no need to leave it in block_int.h Signed-off-by: Emanuele Giuseppe Esposito --- include/block/block_int.h | 3 --- block.c | 4 +++- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/include/block

[PATCH 2/4] include/sysemu/blockdev.h: rename if_name in block_if_name

2021-11-23 Thread Emanuele Giuseppe Esposito
In preparation to next patch, where we export it to be used also in softmmu/vlc.c Signed-off-by: Emanuele Giuseppe Esposito --- blockdev.c | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/blockdev.c b/blockdev.c index b35072644e..1b6ffbbc73 100644 --- a

[PATCH 0/4] block: minor refactoring in preparation to the block layer API split

2021-11-23 Thread Emanuele Giuseppe Esposito
nctions in blockdev.h and block_int.h. Signed-off-by: Emanuele Giuseppe Esposito Emanuele Giuseppe Esposito (4): block_int: make bdrv_backing_overridden static include/sysemu/blockdev.h: rename if_name in block_if_name include/sysemu/blockdev.h: move drive_add and inline drive_def incl

Re: [PATCH v4 00/25] block layer: split block APIs in global state and I/O

2021-11-19 Thread Emanuele Giuseppe Esposito
should have addressed all main comments of this series. Assume that for the others where I did not explicitly answered, I agree and applied your comments. Thank you, Emanuele

Re: [PATCH v4 08/25] block: introduce assert_bdrv_graph_writable

2021-11-19 Thread Emanuele Giuseppe Esposito
On 18/11/2021 16:17, Hanna Reitz wrote: On 18.11.21 10:55, Emanuele Giuseppe Esposito wrote: On 12/11/2021 15:40, Hanna Reitz wrote: On 25.10.21 12:17, Emanuele Giuseppe Esposito wrote: We want to be sure that the functions that write the child and parent list of a bs are under BQL and

Re: [PATCH v4 19/25] block_int-common.h: split function pointers in BlockDriver

2021-11-18 Thread Emanuele Giuseppe Esposito
() specifically creates the create job in the main context (with a TODO comment) and requires block drivers to error out when they encounter a node in a different AioContext. Ok after better reviewing this I agree with you: - .bdrv_co_create_opts is for sure a GS function. It is called by bdrv_create and it is asserted to be under BQL. - .bdrv_co_create should also be a GS, and the easiest thing to do would be to follow the existing TODO and make sure we cannot run it outside the main loop. I think that I will put it as GS, and add the BQL assertion to blockdev_create_run, so that if for some reasons someone tries to do what you did above, will crash because of the assertion, and not because of the aiocontext lock missing. Emanuele

Re: [PATCH v4 08/25] block: introduce assert_bdrv_graph_writable

2021-11-18 Thread Emanuele Giuseppe Esposito
On 18/11/2021 10:55, Emanuele Giuseppe Esposito wrote: On 12/11/2021 15:40, Hanna Reitz wrote: On 25.10.21 12:17, Emanuele Giuseppe Esposito wrote: We want to be sure that the functions that write the child and parent list of a bs are under BQL and drain. BQL prevents from concurrent

Re: [PATCH v4 08/25] block: introduce assert_bdrv_graph_writable

2021-11-18 Thread Emanuele Giuseppe Esposito
On 12/11/2021 15:40, Hanna Reitz wrote: On 25.10.21 12:17, Emanuele Giuseppe Esposito wrote: We want to be sure that the functions that write the child and parent list of a bs are under BQL and drain. BQL prevents from concurrent writings from the GS API, while drains protect from I/O. TODO

Re: [PATCH v4 24/25] job.h: split function pointers in JobDriver

2021-11-17 Thread Emanuele Giuseppe Esposito
On 15/11/2021 16:11, Hanna Reitz wrote: On 25.10.21 12:17, Emanuele Giuseppe Esposito wrote: The job API will be handled separately in another serie. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi ---   include/qemu/job.h | 16   1 file changed, 16

Re: [PATCH v4 20/25] block_int-common.h: assertion in the callers of BlockDriver function pointers

2021-11-17 Thread Emanuele Giuseppe Esposito
On 17/11/2021 13:51, Hanna Reitz wrote: On 17.11.21 12:33, Emanuele Giuseppe Esposito wrote: On 15/11/2021 13:48, Hanna Reitz wrote: On 25.10.21 12:17, Emanuele Giuseppe Esposito wrote: Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi ---   block.c | 17

Re: [PATCH v4 20/25] block_int-common.h: assertion in the callers of BlockDriver function pointers

2021-11-17 Thread Emanuele Giuseppe Esposito
On 15/11/2021 13:48, Hanna Reitz wrote: On 25.10.21 12:17, Emanuele Giuseppe Esposito wrote: Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi ---   block.c | 17 +   1 file changed, 17 insertions(+) diff --git a/block.c b/block.c index 94bff5c757

Re: [PATCH v4 07/25] assertions for block_int global state API

2021-11-16 Thread Emanuele Giuseppe Esposito
On 12/11/2021 14:51, Hanna Reitz wrote: On 25.10.21 12:17, Emanuele Giuseppe Esposito wrote: Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi ---   block.c | 17 +   block/backup.c  |  1 +   block/block-backend.c

Re: [PATCH v4 04/25] include/sysemu/block-backend: split header into I/O and global state (GS) API

2021-11-16 Thread Emanuele Giuseppe Esposito
On 12/11/2021 13:30, Hanna Reitz wrote: On 25.10.21 12:17, Emanuele Giuseppe Esposito wrote: Similarly to the previous patches, split block-backend.h in block-backend-io.h and block-backend-global-state.h In addition, remove "block/block.h" include as it seems it is not necessa

Re: [PATCH v4 02/25] include/block/block: split header into I/O and global state API

2021-11-16 Thread Emanuele Giuseppe Esposito
On 12/11/2021 13:25, Hanna Reitz wrote: On 25.10.21 12:17, Emanuele Giuseppe Esposito wrote: block.h currently contains a mix of functions: some of them run under the BQL and modify the block layer graph, others are instead thread-safe and perform I/O in iothreads. It is not easy to

Re: [PATCH v4 06/25] include/block/block_int: split header into I/O and global state API

2021-11-16 Thread Emanuele Giuseppe Esposito
On 12/11/2021 13:17, Hanna Reitz wrote: On 25.10.21 12:17, Emanuele Giuseppe Esposito wrote: Similarly to the previous patch, split block_int.h in block_int-io.h and block_int-global-state.h block_int-common.h contains the structures shared between the two headers, and the functions that

Re: [PATCH v4 04/25] include/sysemu/block-backend: split header into I/O and global state (GS) API

2021-11-16 Thread Emanuele Giuseppe Esposito
e any iothread using it. I agree with the rest of the comments. Thank you, Emanuele

Re: [PATCH v4 05/25] block/block-backend.c: assertions for block-backend

2021-11-16 Thread Emanuele Giuseppe Esposito
On 12/11/2021 12:01, Hanna Reitz wrote: On 25.10.21 12:17, Emanuele Giuseppe Esposito wrote: All the global state (GS) API functions will check that qemu_in_main_thread() returns true. If not, it means that the safety of BQL cannot be guaranteed, and they need to be moved to I/O. Signed-off

Re: [PATCH v4 03/25] assertions for block global state API

2021-11-15 Thread Emanuele Giuseppe Esposito
tus functions?  Because of the bdrv_filter_or_cow_bs() call? This function is in block.io but has the assertion. I think it's a proper I/O but I forgot to remove the assertion in my various trial&errors. Let me know if you disagree on anything of what I said. Thank you, Emanuele

Re: [PATCH v4 02/25] include/block/block: split header into I/O and global state API

2021-11-15 Thread Emanuele Giuseppe Esposito
. Will fix that. Thank you, Emanuele

[RFC PATCH 3/3] jobs: add job-driver.h

2021-11-04 Thread Emanuele Giuseppe Esposito
job-driver.h contains all functions of job.h that are used by the drivers (JobDriver, BlockJobDriver). These functions are unaware of the job_mutex, so they all take and release the lock internally. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job

[RFC PATCH 1/3] jobs: add job-common.h

2021-11-04 Thread Emanuele Giuseppe Esposito
job-common.h contains all struct and common function that currently are in job.h and will be shared by job-monitor and job-driver in the next commits. Also move job_type() and job_type_str() there, as they are common helper functions. No functional change intended. Signed-off-by: Emanuele

[RFC PATCH 2/3] jobs: add job-monitor.h

2021-11-04 Thread Emanuele Giuseppe Esposito
job-monitor.h contains all functions of job.h that are used by the monitor and essentially all functions that do not define a JobDriver/Blockdriver. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job-monitor.h | 282

[RFC PATCH 0/3] job: split job API in driver and monitor

2021-11-04 Thread Emanuele Giuseppe Esposito
job-driver). I am open to suggestion for alternative names. This serie is based on my previous series "job: replace AioContext lock with job_mutex". Based-on: <20211104145334.1346363-1-eespo...@redhat.com> Emanuele Giuseppe Esposito (3): jobs: add job-common.h jobs: add job-

[RFC PATCH v2 13/14] jobs: add job lock in find_* functions

2021-11-04 Thread Emanuele Giuseppe Esposito
*. Signed-off-by: Emanuele Giuseppe Esposito --- blockdev.c | 9 + job-qmp.c | 8 2 files changed, 17 insertions(+) diff --git a/blockdev.c b/blockdev.c index c5a835d9ed..0bd79757fc 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3327,12 +3327,14 @@ static BlockJob *find_block_job(const

[RFC PATCH v2 12/14] jobs: use job locks and helpers also in the unit tests

2021-11-04 Thread Emanuele Giuseppe Esposito
Add missing job synchronization in the unit tests, with both explicit locks and helpers. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito --- tests/unit/test-bdrv-drain.c | 40 +++--- tests/unit/test-block

[RFC PATCH v2 09/14] jobs: remove aiocontext locks since the functions are under BQL

2021-11-04 Thread Emanuele Giuseppe Esposito
/release. We don't want this, as job_lock can only be *wrapped by* the aiocontext lock, and not vice-versa, to avoid deadlocks. Signed-off-by: Emanuele Giuseppe Esposito --- blockdev.c | 4 job-qmp.c | 4 2 files changed, 8 deletions(-) diff --git a/blockdev.c b/blockdev.c index d9bf8

[RFC PATCH v2 14/14] job.c: enable job lock/unlock and remove Aiocontext locks

2021-11-04 Thread Emanuele Giuseppe Esposito
off-by: Emanuele Giuseppe Esposito --- include/qemu/job.h | 7 --- block/replication.c | 2 + blockdev.c | 62 job-qmp.c| 38 --- job.c|

[RFC PATCH v2 07/14] job.c: move inner aiocontext lock in callbacks

2021-11-04 Thread Emanuele Giuseppe Esposito
. Signed-off-by: Emanuele Giuseppe Esposito --- job.c | 15 +++ 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/job.c b/job.c index ce5066522f..7856fa734b 100644 --- a/job.c +++ b/job.c @@ -170,7 +170,6 @@ static void job_txn_del_job(Job *job) static int job_txn_apply(Job

[RFC PATCH v2 06/14] job.c: make job_event_* functions static

2021-11-04 Thread Emanuele Giuseppe Esposito
job_event_* functions can all be static, as they are not used outside job.c. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job.h | 6 -- job.c | 12 ++-- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/include/qemu/job.h b/include/qemu

[RFC PATCH v2 11/14] block_job_query: remove atomic read

2021-11-04 Thread Emanuele Giuseppe Esposito
Not sure what the atomic here was supposed to do, since job.busy is protected by the job lock. Since the whole function will be called under job_mutex, just remove the atomic. Signed-off-by: Emanuele Giuseppe Esposito --- blockjob.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions

[RFC PATCH v2 04/14] job.h: define unlocked functions

2021-11-04 Thread Emanuele Giuseppe Esposito
in job.c). Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job.h | 21 +++ blockjob.c | 20 --- job.c | 88 -- 3 files changed

[RFC PATCH v2 05/14] block/mirror.c: use of job helpers in drivers to avoid TOC/TOU

2021-11-04 Thread Emanuele Giuseppe Esposito
Once job lock is used and aiocontext is removed, mirror has to perform job operations under the same critical section, using the helpers prepared in previous commit. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito --- block

[RFC PATCH v2 10/14] jobs: protect jobs with job_lock/unlock

2021-11-04 Thread Emanuele Giuseppe Esposito
macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 6 ++ block/replication.c | 4 + blockdev.c | 13 +++ blockjob.c | 37 +++- job-qmp.c | 4 + job.c | 201 ++-- monitor/qmp

[RFC PATCH v2 01/14] job.c: make job_lock/unlock public

2021-11-04 Thread Emanuele Giuseppe Esposito
, introduce *nop* lock/unlock functions and macros. Once everything is protected by jobs, we can add the mutex and remove the aiocontext. Since job_mutex is already being used, add static _job_{lock/unlock}. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- include/qemu/job.h | 18

[RFC PATCH v2 00/14] job: replace AioContext lock with job_mutex

2021-11-04 Thread Emanuele Giuseppe Esposito
mpty job_{lock/unlock} and JOB_LOCK_GUARD/WITH_JOB_LOCK_GUARD to avoid deadlocks and simplify the reviewer job Emanuele Giuseppe Esposito (14): job.c: make job_lock/unlock public job.h: categorize fields in struct Job job.h: define locked functions job.h: define unlocked functions block

[RFC PATCH v2 08/14] aio-wait.h: introduce AIO_WAIT_WHILE_UNLOCKED

2021-11-04 Thread Emanuele Giuseppe Esposito
Same as AIO_WAIT_WHILE macro, but if we are in the Main loop do not release and then acquire ctx_ 's aiocontext. Once all Aiocontext locks go away, this macro will replace AIO_WAIT_WHILE. Signed-off-by: Emanuele Giuseppe Esposito --- include/block/aio-wait.h | 15 +++ 1

[RFC PATCH v2 02/14] job.h: categorize fields in struct Job

2021-11-04 Thread Emanuele Giuseppe Esposito
Categorize the fields in struct Job to understand which ones need to be protected by the job mutex and which don't. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job.h | 57 +++--- 1 file changed, 34 insertions(+), 23 deletions(-) diff

[RFC PATCH v2 03/14] job.h: define locked functions

2021-11-04 Thread Emanuele Giuseppe Esposito
: Emanuele Giuseppe Esposito --- include/qemu/job.h | 66 ++ job.c | 18 +++-- 2 files changed, 77 insertions(+), 7 deletions(-) diff --git a/include/qemu/job.h b/include/qemu/job.h index f7036ac6b3..c7a6bcad1b 100644 --- a/include

[PATCH v1] job.c: add missing notifier initialization

2021-11-03 Thread Emanuele Giuseppe Esposito
It seems that on_idle list is not properly initialized like the other notifiers. Signed-off-by: Emanuele Giuseppe Esposito --- job.c | 1 + 1 file changed, 1 insertion(+) diff --git a/job.c b/job.c index dbfa67bb0a..54db80df66 100644 --- a/job.c +++ b/job.c @@ -352,6 +352,7 @@ void *job_create

Re: [RFC PATCH 11/15] jobs: remove aiocontext locks since the functions are under BQL

2021-11-03 Thread Emanuele Giuseppe Esposito
obs fields. - job_query_single: same as block_job_query, calls progress_get_snapshot and accesses job fields. So yes, I am positive that here removing the aiocontext lock does not break anything. Thank you, Emanuele

Re: [RFC PATCH 00/15] job: replace AioContext lock with job_mutex

2021-11-02 Thread Emanuele Giuseppe Esposito
On 02/11/2021 14:08, Vladimir Sementsov-Ogievskiy wrote: 29.10.2021 19:38, Emanuele Giuseppe Esposito wrote: In this series, we want to remove the AioContext lock and instead use the already existent job_mutex to protect the job structures and list. This is part of the work to get rid of

[RFC PATCH 13/15] jobs: use job locks and helpers also in the unit tests

2021-10-29 Thread Emanuele Giuseppe Esposito
Add missing job synchronization in the unit tests, with both explicit locks and helpers. Signed-off-by: Emanuele Giuseppe Esposito --- tests/unit/test-bdrv-drain.c | 40 +++--- tests/unit/test-block-iothread.c | 4 +++ tests/unit/test-blockjob-txn.c | 10 ++ tests

[RFC PATCH 11/15] jobs: remove aiocontext locks since the functions are under BQL

2021-10-29 Thread Emanuele Giuseppe Esposito
/release. We don't want this, as job_lock can only be *wrapped by* the aiocontext lock, and not vice-versa, to avoid deadlocks. Signed-off-by: Emanuele Giuseppe Esposito --- blockdev.c | 4 job-qmp.c | 4 2 files changed, 8 deletions(-) diff --git a/blockdev.c b/blockdev.c index d9bf8

[RFC PATCH 07/15] job-driver.h: add helper functions

2021-10-29 Thread Emanuele Giuseppe Esposito
: Emanuele Giuseppe Esposito --- include/qemu/job-driver.h | 21 +++ blockjob.c| 20 --- job.c | 116 +- 3 files changed, 135 insertions(+), 22 deletions(-) diff --git a/include/qemu/job-driver.h b/include/qemu/job

[RFC PATCH 12/15] jobs: protect jobs with job_lock/unlock

2021-10-29 Thread Emanuele Giuseppe Esposito
his! However, adding an additional unlock after job_ref() and lock under the aiocontext acquire seems unnecessary, as it will all go away in the next commits. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 6 ++ block/mirror.c | 8 +- block/replication.c |

[RFC PATCH 08/15] job.c: minor adjustments in preparation to job-driver

2021-10-29 Thread Emanuele Giuseppe Esposito
job_event_* functions can be all static, as they are not used outside job.c Add also missing notifier initialization for the on_idle list in job_create(). Signed-off-by: Emanuele Giuseppe Esposito --- job.c | 13 +++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a

[RFC PATCH 15/15] jobs: remove all unnecessary AioContext locks

2021-10-29 Thread Emanuele Giuseppe Esposito
outside the aiocontext lock by adding unlock/lock pairs. There is only one JobDriver callback, ->free() that assumes that the aiocontext lock is held (because it calls bdrv_unref), so for now keep that under aiocontext lock. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/

[RFC PATCH 14/15] jobs: add missing job locks to replace aiocontext lock

2021-10-29 Thread Emanuele Giuseppe Esposito
the job API, many of which assume the aiocontext lock is held. To keep this commit simple, handle the rest of the aiocontext removal in the next commit. Signed-off-by: Emanuele Giuseppe Esposito --- blockdev.c | 52 ++-- job-qmp.c | 46

[RFC PATCH 06/15] jobs: add job-driver.h

2021-10-29 Thread Emanuele Giuseppe Esposito
job-driver.h contains all functions of job.h that are used by the drivers (JobDriver, BlockJobDriver). These functions are unaware of the job_mutex, so they all will take and release the lock internally. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu

[RFC PATCH 09/15] job.c: move inner aiocontext lock in callbacks

2021-10-29 Thread Emanuele Giuseppe Esposito
. Signed-off-by: Emanuele Giuseppe Esposito --- job.c | 13 + 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/job.c b/job.c index 88d911f2d7..eb6d321960 100644 --- a/job.c +++ b/job.c @@ -153,7 +153,6 @@ static void job_txn_del_job(Job *job) static int job_txn_apply(Job

[RFC PATCH 10/15] aio-wait.h: introduce AIO_WAIT_WHILE_UNLOCKED

2021-10-29 Thread Emanuele Giuseppe Esposito
Same as AIO_WAIT_WHILE macro, but if we are in the Main loop do not release and then acquire ctx_ 's aiocontext. Once all Aiocontext locks go away, this macro will replace AIO_WAIT_WHILE. Signed-off-by: Emanuele Giuseppe Esposito --- include/block/aio-wait.h | 15 +++ 1

[RFC PATCH 05/15] job-monitor.h: define the job monitor API

2021-10-29 Thread Emanuele Giuseppe Esposito
These functions assume that the job lock is held by the caller, to avoid TOC/TOU conditions. Introduce also additional helpers that define _locked functions (useful when the job_mutex is globally applied). Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job-monitor.h | 61

[RFC PATCH 02/15] job.c: make job_lock/unlock public

2021-10-29 Thread Emanuele Giuseppe Esposito
job mutex will be used to protect the job struct elements and list, replacing AioContext locks. Right now use a shared lock for all jobs, in order to keep things simple. Once the AioContext lock is gone, we can introduce per-job locks. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu

[RFC PATCH 03/15] job-common.h: categorize fields in struct Job

2021-10-29 Thread Emanuele Giuseppe Esposito
Categorize the fields in struct Job to understand which need to be protected by the job muutex and which not. Also move job_type() and job_type_str() there, as they are common helper functions. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job-common.h | 62

[RFC PATCH 01/15] jobs: add job-common.h

2021-10-29 Thread Emanuele Giuseppe Esposito
job-common.h contains all struct and common function that currently are in job.h and will be shared by job-monitor and job-driver in the next commits. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job-common.h | 300

[RFC PATCH 04/15] jobs: add job-monitor.h

2021-10-29 Thread Emanuele Giuseppe Esposito
-by: Emanuele Giuseppe Esposito --- include/qemu/job-monitor.h | 220 + include/qemu/job.h | 180 +- 2 files changed, 221 insertions(+), 179 deletions(-) create mode 100644 include/qemu/job-monitor.h diff --git a/include/qemu

[RFC PATCH 00/15] job: replace AioContext lock with job_mutex

2021-10-29 Thread Emanuele Giuseppe Esposito
and qtests (x86_64). This serie is based on my previous series "block layer: split block APIs in global state and I/O". Based-on: <20211025101735.2060852-1-eespo...@redhat.com> Signed-off-by: Emanuele Giuseppe Esposito Emanuele Giuseppe Esposito (15): jobs: add job-common.h j

Re: [PATCH v4 02/25] include/block/block: split header into I/O and global state API

2021-10-25 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- block.c| 3 + block/meson.build | 7 +- include/block/block-common.h | 389 + Can this patch be split in 3? (first) include/block/block

[PATCH v4 24/25] job.h: split function pointers in JobDriver

2021-10-25 Thread Emanuele Giuseppe Esposito
The job API will be handled separately in another serie. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- include/qemu/job.h | 16 1 file changed, 16 insertions(+) diff --git a/include/qemu/job.h b/include/qemu/job.h index 6e67b6977f..7e9e59f4b8

<    5   6   7   8   9   10   11   12   13   14   >