Re: [PATCH v3 3/3] qapi: deprecate drive-backup

2021-11-04 Thread Eric Blake
On Wed, Nov 03, 2021 at 02:29:12PM +0100, Vladimir Sementsov-Ogievskiy wrote: > Modern way is using blockdev-add + blockdev-backup, which provides a > lot more control on how target is opened. > > > Signed-off-by: Vladimir Sementsov-Ogievskiy > Reviewed-by: Kashyap Chamarthy > --- >

Re: [PATCH 12/15] hw/nvme: Initialize capability structures for primary/secondary controllers

2021-11-04 Thread Łukasz Gieryk
On Wed, Nov 03, 2021 at 01:07:31PM +0100, Klaus Jensen wrote: > On Oct 7 18:24, Lukasz Maniak wrote: > > From: Łukasz Gieryk > > > > With two new properties (sriov_max_vi_per_vf, sriov_max_vq_per_vf) one > > can configure the maximum number of virtual queues and interrupts > > assignable to a

Re: [PATCH v8 4/8] qmp: add QMP command x-debug-virtio-status

2021-11-04 Thread Markus Armbruster
Jonah Palmer writes: > From: Laurent Vivier > > This new command shows the status of a VirtIODevice, including > its corresponding vhost device status (if active). > > Next patch will improve output by decoding feature bits, including > vhost device's feature bits (backend, protocol, acked, and

[RFC PATCH 3/3] jobs: add job-driver.h

2021-11-04 Thread Emanuele Giuseppe Esposito
job-driver.h contains all functions of job.h that are used by the drivers (JobDriver, BlockJobDriver). These functions are unaware of the job_mutex, so they all take and release the lock internally. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito ---

[RFC PATCH 0/3] job: split job API in driver and monitor

2021-11-04 Thread Emanuele Giuseppe Esposito
In this series, we split the job API in two headers: job-driver.h and job-monitor.h. As explained in job.c, job-monitor are the functions mainly used by the monitor, and require consistency between the search of a specific job (job_get) and the actual operation/action on it (e.g. job_user_cancel).

[RFC PATCH 1/3] jobs: add job-common.h

2021-11-04 Thread Emanuele Giuseppe Esposito
job-common.h contains all struct and common function that currently are in job.h and will be shared by job-monitor and job-driver in the next commits. Also move job_type() and job_type_str() there, as they are common helper functions. No functional change intended. Signed-off-by: Emanuele

[RFC PATCH 2/3] jobs: add job-monitor.h

2021-11-04 Thread Emanuele Giuseppe Esposito
job-monitor.h contains all functions of job.h that are used by the monitor and essentially all functions that do not define a JobDriver/Blockdriver. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job-monitor.h | 282

Re: [PATCH v8 3/8] qmp: add QMP command x-debug-query-virtio

2021-11-04 Thread Markus Armbruster
Jonah Palmer writes: > From: Laurent Vivier > > This new command lists all the instances of VirtIODevice with > their QOM paths and virtio type/name. > > Signed-off-by: Jonah Palmer > --- > hw/virtio/meson.build | 2 ++ > hw/virtio/virtio-stub.c| 14 ++ > hw/virtio/virtio.c

[RFC PATCH v2 12/14] jobs: use job locks and helpers also in the unit tests

2021-11-04 Thread Emanuele Giuseppe Esposito
Add missing job synchronization in the unit tests, with both explicit locks and helpers. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito --- tests/unit/test-bdrv-drain.c | 40 +++---

[RFC PATCH v2 14/14] job.c: enable job lock/unlock and remove Aiocontext locks

2021-11-04 Thread Emanuele Giuseppe Esposito
Change the job_{lock/unlock} and macros to use job_mutex. Now that they are not nop anymore, remove the aiocontext to avoid deadlocks. Therefore: - when possible, remove completely the aiocontext lock/unlock pair - if it is used by some other functions too, reduce the locking section as much as

[RFC PATCH v2 07/14] job.c: move inner aiocontext lock in callbacks

2021-11-04 Thread Emanuele Giuseppe Esposito
Instead of having the lock in job_tnx_apply, move it inside in the callback. This will be helpful for next commits, when we introduce job_lock/unlock pairs. job_transition_to_pending() and job_needs_finalize() do not need to be protected by the aiocontext lock. No functional change intended.

[RFC PATCH v2 10/14] jobs: protect jobs with job_lock/unlock

2021-11-04 Thread Emanuele Giuseppe Esposito
Introduce the job locking mechanism through the whole job API, following the comments and requirements of job-monitor (assume lock is held) and job-driver (lock is not held). job_{lock/unlock} is independent from _job_{lock/unlock}. Note: at this stage, job_{lock/unlock} and job lock guard

[RFC PATCH v2 11/14] block_job_query: remove atomic read

2021-11-04 Thread Emanuele Giuseppe Esposito
Not sure what the atomic here was supposed to do, since job.busy is protected by the job lock. Since the whole function will be called under job_mutex, just remove the atomic. Signed-off-by: Emanuele Giuseppe Esposito --- blockjob.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)

[RFC PATCH v2 09/14] jobs: remove aiocontext locks since the functions are under BQL

2021-11-04 Thread Emanuele Giuseppe Esposito
In preparation to the job_lock/unlock patch, remove these aiocontext locks. The main reason these two locks are removed here is because they are inside a loop iterating on the jobs list. Once the job_lock is added, it will have to protect the whole loop, wrapping also the aiocontext

[RFC PATCH v2 05/14] block/mirror.c: use of job helpers in drivers to avoid TOC/TOU

2021-11-04 Thread Emanuele Giuseppe Esposito
Once job lock is used and aiocontext is removed, mirror has to perform job operations under the same critical section, using the helpers prepared in previous commit. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*. Signed-off-by: Emanuele Giuseppe Esposito ---

[RFC PATCH v2 08/14] aio-wait.h: introduce AIO_WAIT_WHILE_UNLOCKED

2021-11-04 Thread Emanuele Giuseppe Esposito
Same as AIO_WAIT_WHILE macro, but if we are in the Main loop do not release and then acquire ctx_ 's aiocontext. Once all Aiocontext locks go away, this macro will replace AIO_WAIT_WHILE. Signed-off-by: Emanuele Giuseppe Esposito --- include/block/aio-wait.h | 15 +++ 1 file

[RFC PATCH v2 13/14] jobs: add job lock in find_* functions

2021-11-04 Thread Emanuele Giuseppe Esposito
Both blockdev.c and job-qmp.c have TOC/TOU conditions, because they first search for the job and then perform an action on it. Therefore, we need to do the search + action under the same job mutex critical section. Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*.

[RFC PATCH v2 06/14] job.c: make job_event_* functions static

2021-11-04 Thread Emanuele Giuseppe Esposito
job_event_* functions can all be static, as they are not used outside job.c. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job.h | 6 -- job.c | 12 ++-- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/include/qemu/job.h

[RFC PATCH v2 04/14] job.h: define unlocked functions

2021-11-04 Thread Emanuele Giuseppe Esposito
All these functions assume that the lock is not held, and acquire it internally. These functions will be useful when job_lock is globally applied, as they will allow callers to access the job struct fields without worrying about the job lock. Update also the comments in blockjob.c (and move them

[RFC PATCH v2 01/14] job.c: make job_lock/unlock public

2021-11-04 Thread Emanuele Giuseppe Esposito
job mutex will be used to protect the job struct elements and list, replacing AioContext locks. Right now use a shared lock for all jobs, in order to keep things simple. Once the AioContext lock is gone, we can introduce per-job locks. To simplify the switch from aiocontext to job lock,

[RFC PATCH v2 02/14] job.h: categorize fields in struct Job

2021-11-04 Thread Emanuele Giuseppe Esposito
Categorize the fields in struct Job to understand which ones need to be protected by the job mutex and which don't. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job.h | 57 +++--- 1 file changed, 34 insertions(+), 23 deletions(-) diff --git

[RFC PATCH v2 00/14] job: replace AioContext lock with job_mutex

2021-11-04 Thread Emanuele Giuseppe Esposito
In this series, we want to remove the AioContext lock and instead use the already existent job_mutex to protect the job structures and list. This is part of the work to get rid of AioContext lock usage in favour of smaller granularity locks. In order to simplify reviewer's job, job lock/unlock

[RFC PATCH v2 03/14] job.h: define locked functions

2021-11-04 Thread Emanuele Giuseppe Esposito
These functions assume that the job lock is held by the caller, to avoid TOC/TOU conditions. Introduce also additional helpers that define _locked functions (useful when the job_mutex is globally applied). Note: at this stage, job_{lock/unlock} and job lock guard macros are *nop*.

Re: [PATCH] file-posix: Fix alignment after reopen changing O_DIRECT

2021-11-04 Thread Kevin Wolf
Am 04.11.2021 um 15:20 hat Stefano Garzarella geschrieben: > On Thu, Nov 04, 2021 at 12:31:09PM +0100, Kevin Wolf wrote: > > At the end of a reopen, we already call bdrv_refresh_limits(), which > > should update bs->request_alignment according to the new file > > descriptor. However,

Re: [PATCH v2 0/3] virtio: increase VIRTQUEUE_MAX_SIZE to 32k

2021-11-04 Thread Christian Schoenebeck
On Mittwoch, 3. November 2021 12:33:33 CET Stefan Hajnoczi wrote: > On Mon, Nov 01, 2021 at 09:29:26PM +0100, Christian Schoenebeck wrote: > > On Donnerstag, 28. Oktober 2021 11:00:48 CET Stefan Hajnoczi wrote: > > > On Mon, Oct 25, 2021 at 05:03:25PM +0200, Christian Schoenebeck wrote: > > > > On

Re: [PATCH] file-posix: Fix alignment after reopen changing O_DIRECT

2021-11-04 Thread Stefano Garzarella
On Thu, Nov 04, 2021 at 12:31:09PM +0100, Kevin Wolf wrote: At the end of a reopen, we already call bdrv_refresh_limits(), which should update bs->request_alignment according to the new file descriptor. However, raw_probe_alignment() relies on s->needs_alignment and just uses 1 if it isn't set.

Re: [PATCH 05/15] hw/nvme: Add support for SR-IOV

2021-11-04 Thread Lukasz Maniak
On Tue, Nov 02, 2021 at 06:33:31PM +0100, Lukasz Maniak wrote: > On Tue, Nov 02, 2021 at 03:33:15PM +0100, Klaus Jensen wrote: > > On Oct 7 18:23, Lukasz Maniak wrote: > > > This patch implements initial support for Single Root I/O Virtualization > > > on an NVMe device. > > > > > > Essentially,

Re: [PATCH 0/7] block: Attempt on fixing 030-reported errors

2021-11-04 Thread Hanna Reitz
On 04.11.21 12:58, Kevin Wolf wrote: Am 04.11.2021 um 11:38 hat Hanna Reitz geschrieben: (2A) bdrv_replace_child_noperm() should immediately set bs->file or bs->backing to NULL when it sets bs->{file,backing}->bs to NULL. It should also immediately remove any BdrvChild with .bs ==

Re: [PATCH] file-posix: Fix alignment after reopen changing O_DIRECT

2021-11-04 Thread Hanna Reitz
On 04.11.21 12:31, Kevin Wolf wrote: At the end of a reopen, we already call bdrv_refresh_limits(), which should update bs->request_alignment according to the new file descriptor. However, raw_probe_alignment() relies on s->needs_alignment and just uses 1 if it isn't set. We neglected to update

Re: [PATCH 0/7] block: Attempt on fixing 030-reported errors

2021-11-04 Thread Kevin Wolf
Am 04.11.2021 um 11:38 hat Hanna Reitz geschrieben: > (2A) bdrv_replace_child_noperm() should immediately set bs->file or > bs->backing to NULL when it sets bs->{file,backing}->bs to NULL. > It should also immediately remove any BdrvChild with .bs == NULL > from the parent’s

[PATCH] file-posix: Fix alignment after reopen changing O_DIRECT

2021-11-04 Thread Kevin Wolf
At the end of a reopen, we already call bdrv_refresh_limits(), which should update bs->request_alignment according to the new file descriptor. However, raw_probe_alignment() relies on s->needs_alignment and just uses 1 if it isn't set. We neglected to update this field, so starting with

[PATCH 6/7] block: Let replace_child_noperm free children

2021-11-04 Thread Hanna Reitz
In most of the block layer, especially when traversing down from other BlockDriverStates, we assume that BdrvChild.bs can never be NULL. When it becomes NULL, it is expected that the corresponding BdrvChild pointer also becomes NULL and the BdrvChild object is freed. Therefore, once

[PATCH 4/7] block: Drop detached child from ignore list

2021-11-04 Thread Hanna Reitz
bdrv_attach_child_common_abort() restores the parent's AioContext. To do so, the child (which was supposed to be attached, but is now detached again by this abort handler) is added to the ignore list for the AioContext changing functions. However, since we modify a BDS's children list in the

[PATCH 5/7] block: Pass BdrvChild ** to replace_child_noperm

2021-11-04 Thread Hanna Reitz
bdrv_replace_child_noperm() modifies BdrvChild.bs, and can potentially set it to NULL. That is dangerous, because BDS parents generally assume that their children's .bs pointer is never NULL. We therefore want to let bdrv_replace_child_noperm() set the corresponding BdrvChild pointer to NULL,

[PATCH 7/7] iotests/030: Unthrottle parallel jobs in reverse

2021-11-04 Thread Hanna Reitz
See the comment for why this is necessary. Signed-off-by: Hanna Reitz --- tests/qemu-iotests/030 | 11 ++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030 index 5fb65b4bef..567bf1da67 100755 --- a/tests/qemu-iotests/030 +++

[PATCH 3/7] block: Unite remove_empty_child and child_free

2021-11-04 Thread Hanna Reitz
Now that bdrv_remove_empty_child() no longer removes the child from the parent's children list but only checks that it is not in such a list, it is only a wrapper around bdrv_child_free() that checks that the child is empty and unused. That should apply to all children that we free, so put those

[PATCH 2/7] block: Manipulate children list in .attach/.detach

2021-11-04 Thread Hanna Reitz
The children list is specific to BDS parents. We should not modify it in the general children modification code, but let BDS parents deal with it in their .attach() and .detach() methods. This also has the advantage that a BdrvChild is removed from the children list before its .bs pointer can

[PATCH 0/7] block: Attempt on fixing 030-reported errors

2021-11-04 Thread Hanna Reitz
Hi, I’ve tried to investigate what causes the iotest 030 to fail. Here’s what I found: (1) stream_prepare() gets the base node by looking up the node below above_base. It then invokes bdrv_cor_filter_drop(), before we actually use the base node. bdrv_cor_filter_drop() modifies the

[PATCH 1/7] stream: Traverse graph after modification

2021-11-04 Thread Hanna Reitz
bdrv_cor_filter_drop() modifies the block graph. That means that other parties can also modify the block graph before it returns. Therefore, we cannot assume that the result of a graph traversal we did before remains valid afterwards. We should thus fetch `base` and `unfiltered_base` afterwards

Re: [PATCH v1] job.c: add missing notifier initialization

2021-11-04 Thread Stefan Hajnoczi
On Wed, Nov 03, 2021 at 12:21:55PM -0400, Emanuele Giuseppe Esposito wrote: > It seems that on_idle list is not properly initialized like > the other notifiers. > > Signed-off-by: Emanuele Giuseppe Esposito > --- > job.c | 1 + > 1 file changed, 1 insertion(+) Reviewed-by: Stefan Hajnoczi

[PATCH v4 3/3] qapi: deprecate drive-backup

2021-11-04 Thread Markus Armbruster
From: Vladimir Sementsov-Ogievskiy Modern way is using blockdev-add + blockdev-backup, which provides a lot more control on how target is opened. As example of drive-backup problems consider the following: User of drive-backup expects that target will be opened in the same cache and aio mode

[PATCH v4 1/3] docs/block-replication: use blockdev-backup

2021-11-04 Thread Markus Armbruster
From: Vladimir Sementsov-Ogievskiy We are going to deprecate drive-backup, so don't mention it here. Moreover, blockdev-backup seems more correct in the context. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Signed-off-by: Markus Armbruster ---

[PATCH v4 0/3] qapi & doc: deprecate drive-backup

2021-11-04 Thread Markus Armbruster
See 03 commit message for details. 01-02 are preparation docs update. v4: deprecate drive-backup transaction by squashing [PATCH v4 5/5] block: Deprecate transaction type drive-backup Message-Id: <20211025042405.3762351-6-arm...@redhat.com> into PATCH 3 v3: wording fix-ups

[PATCH v4 2/3] docs/interop/bitmaps: use blockdev-backup

2021-11-04 Thread Markus Armbruster
From: Vladimir Sementsov-Ogievskiy We are going to deprecate drive-backup, so use modern interface here. In examples where target image creation is shown, show blockdev-add as well. If target creation omitted, omit blockdev-add as well. Reviewed-by: Kashyap Chamarthy Signed-off-by: Vladimir