Re: [PATCH v5 19/20] job.c: enable job lock/unlock and remove Aiocontext locks

2022-03-08 Thread Stefan Hajnoczi
On Tue, Feb 08, 2022 at 09:35:12AM -0500, Emanuele Giuseppe Esposito wrote: > diff --git a/include/qemu/job.h b/include/qemu/job.h > index ca46e46f5b..574110a1f2 100644 > --- a/include/qemu/job.h > +++ b/include/qemu/job.h > @@ -75,11 +75,14 @@ typedef struct Job { > ProgressMeter progress; >

Re: [PATCH v5 16/20] commit and mirror: create new nodes using bdrv_get_aio_context, and not the job aiocontext

2022-03-08 Thread Stefan Hajnoczi
On Tue, Feb 08, 2022 at 09:35:09AM -0500, Emanuele Giuseppe Esposito wrote: > We are always using the given bs AioContext, so there is no need > to take the job ones (which is identical anyways). > This also reduces the point we need to check when protecting > job.aio_context field. > >

Re: [PATCH v5 17/20] job: detect change of aiocontext within job coroutine

2022-03-08 Thread Stefan Hajnoczi
On Tue, Feb 08, 2022 at 09:35:10AM -0500, Emanuele Giuseppe Esposito wrote: > From: Paolo Bonzini > > We want to make sure access of job->aio_context is always done > under either BQL or job_mutex. The problem is that using > aio_co_enter(job->aiocontext, job->co) in job_start and job_enter_cond

Re: [PATCH v5 20/20] block_job_query: remove atomic read

2022-03-08 Thread Stefan Hajnoczi
On Tue, Feb 08, 2022 at 09:35:13AM -0500, Emanuele Giuseppe Esposito wrote: > Not sure what the atomic here was supposed to do, since job.busy > is protected by the job lock. Since the whole function > is called under job_mutex, just remove the atomic. > > Signed-off-by: Emanuele Giuseppe

Re: [PATCH v3 3/5] iotests: Remove explicit checks for qemu_img() == 0

2022-03-08 Thread Eric Blake
On Mon, Mar 07, 2022 at 08:57:26PM -0500, John Snow wrote: > qemu_img() returning zero ought to be the rule, not the > exception. Remove all explicit checks against the condition in > preparation for making non-zero returns an Exception. > > Signed-off-by: John Snow > --- Reviewed-by: Eric

Re: [PATCH v5 18/20] jobs: protect job.aio_context with BQL and job_mutex

2022-03-08 Thread Stefan Hajnoczi
On Tue, Feb 08, 2022 at 09:35:11AM -0500, Emanuele Giuseppe Esposito wrote: > static AioContext *child_job_get_parent_aio_context(BdrvChild *c) > { > BlockJob *job = c->opaque; > +assert(qemu_in_main_thread()); > > return job->job.aio_context; > } It's not clear to me that

Re: [PATCH v3 3/5] iotests: Remove explicit checks for qemu_img() == 0

2022-03-08 Thread John Snow
On Tue, Mar 8, 2022, 10:16 AM Eric Blake wrote: > On Mon, Mar 07, 2022 at 08:57:26PM -0500, John Snow wrote: > > qemu_img() returning zero ought to be the rule, not the > > exception. Remove all explicit checks against the condition in > > preparation for making non-zero returns an Exception. >

[PATCH 13/14] iotests: make qemu_img_log() check log level

2022-03-08 Thread John Snow
Improve qemu_img_log() to actually check if logging is turned on. If it isn't, revert to the behavior of qemu_img(). This is done so that there really is no way to avoid scrutinizing qemu-ing subprocess calls by accident. Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 19

[PATCH 06/14] iotests: change supports_quorum to use qemu_img

2022-03-08 Thread John Snow
Similar to other recent changes: use the qemu_img() invocation that supports throwing loud, nasty exceptions when it fails for surprising reasons. (Why would "--help" ever fail? I don't know, but eliminating *all* calls to qemu-img that do not go through qemu_img() is my goal, so qemu_img_pipe()

[PATCH 00/14] iotests: ensure all qemu-img calls are either checked or logged

2022-03-08 Thread John Snow
Based-On: 20220308015728.1269649-1-js...@redhat.com Hi, this series ensures all calls to qemu-img ultimately go through qemu_img(). After the previous series, qemu_img() is a function that defaults to raising a VerboseProcessError exception when qemu-img returns a non-zero exit code. After this

[PATCH 01/14] iotests: add qemu_img_json()

2022-03-08 Thread John Snow
qemu_img_json() is a new helper built on top of qemu_img() that tries to pull a valid JSON document out of the stdout stream. In the event that the return code is negative (the program crashed), or the code is greater than zero and did not produce valid JSON output, the VerboseProcessError raised

[PATCH 04/14] iotests/remove-bitmap-from-backing: use qemu_img_info()

2022-03-08 Thread John Snow
This removes two more usages of qemu_img_pipe() and replaces them with calls to qemu_img(), which provides better diagnostic information on failure. Signed-off-by: John Snow --- tests/qemu-iotests/tests/remove-bitmap-from-backing | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff

[PATCH 02/14] iotests: use qemu_img_json() when applicable

2022-03-08 Thread John Snow
qemu_img_json() gives better diagnostic information on failure. Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 546b142a6c..7b37938d45

[PATCH 12/14] iotests: remove qemu_img_pipe_and_status()

2022-03-08 Thread John Snow
With the exceptional 'create' calls removed in the prior commit, change qemu_img_log() and img_info_log() to call qemu_img() directly instead. In keeping with the spirit of diff-based tests, allow these calls to qemu_img() to return an unchecked non-zero status code -- because any error we'd see

[PATCH 11/14] iotests: replace qemu_img_log('create', ...) calls

2022-03-08 Thread John Snow
qemu_img_log() calls into qemu_img_pipe(), which always removes output for 'create' commands on success anyway. Replace all of these calls to the simpler qemu_img_create(...) which doesn't log, but raises a detailed exception object on failure instead. Blank lines are removed from output files

[PATCH 08/14] iotests/149: Remove qemu_img_pipe() call

2022-03-08 Thread John Snow
qemu_img_pipe calls blank their output when the command being run is a 'create' call and the command succeeds. Thus, the normative output for this command in iotest 149 is to print a blank line. We can remove the logging from this invocation and use a checked invocation, but we still need to

[PATCH 07/14] iotests: replace unchecked calls to qemu_img_pipe()

2022-03-08 Thread John Snow
qemu_img_pipe() discards the return code from qemu-img in favor of returning just its output. Some tests using this function don't save, log, or check the output either, though, which is unsafe. Replace all of these calls with a checked version. Tests affected are 194, 202, 203, 234, 262, and

[PATCH 05/14] iotests: add qemu_img_map() function

2022-03-08 Thread John Snow
Add a qemu_img_map() function by analogy with qemu_img_measure(), qemu_img_check(), and qemu_img_info() that all return JSON information. Replace calls to qemu_img_pipe('map', '--output=json', ...) with this new function, which provides better diagnostic information on failure. Signed-off-by:

[PATCH 03/14] iotests: add qemu_img_info()

2022-03-08 Thread John Snow
Add qemu_img_info() by analogy with qemu_img_measure() and qemu_img_check(). Modify image_size() to use this function instead to take advantage of the better diagnostic information on failure provided (ultimately) by qemu_img(). Signed-off-by: John Snow --- tests/qemu-iotests/065| 5

[PATCH 10/14] iotests: use qemu_img() in has_working_luks()

2022-03-08 Thread John Snow
Admittedly a mostly lateral move, but qemu_img() is essentially the replacement for qemu_img_pipe_and_status(). It will give slightly better diagnostics on crash. Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 18 +- 1 file changed, 9 insertions(+), 9 deletions(-)

[PATCH 09/14] iotests: remove remaining calls to qemu_img_pipe()

2022-03-08 Thread John Snow
As part of moving all python iotest invocations of qemu-img onto a single qemu_img() implementation, remove a few lingering uses of qemu_img_pipe() from outside of iotests.py itself. Several cases here rely on the knowledge that qemu_img_pipe() suppresses *all* output on a successful case when

[PATCH 14/14] iotests: make img_info_log() call qemu_img_log()

2022-03-08 Thread John Snow
Add configurable filters to qemu_img_log(), and re-write img_info_log() to call into qemu_img_log() with a custom filter instead. After this patch, every last call to qemu_img() is now guaranteed to either have its return code checked for zero, OR have its output actually visibly logged

Re: [PULL 00/11] Python patches

2022-03-08 Thread Peter Maydell
On Mon, 7 Mar 2022 at 22:15, John Snow wrote: > > The following changes since commit b49872aa8fc0f3f5a3036cc37aa2cb5c92866f33: > > Merge remote-tracking branch > 'remotes/hreitz-gitlab/tags/pull-block-2022-03-07' into staging (2022-03-07 > 17:14:09 +) > > are available in the Git

[PATCH v2] block/nbd.c: Fixed IO request coroutine not being wakeup when kill NBD server

2022-03-08 Thread Rao Lei
During the IO stress test, the IO request coroutine has a probability that is can't be awakened when the NBD server is killed. The GDB stack is as follows: (gdb) bt 0 0x7f2ff990cbf6 in __ppoll (fds=0x55575de85000, nfds=1, timeout=, sigmask=0x0) at ../sysdeps/unix/sysv/linux/ppoll.c:44 1

Re: [PATCH] block/nbd.c: Fixed IO request coroutine not being wakeup when kill NBD server.

2022-03-08 Thread Rao, Lei
On 3/3/2022 10:05 PM, Rao, Lei wrote: On 3/3/2022 5:25 PM, Vladimir Sementsov-Ogievskiy wrote: 03.03.2022 05:21, Rao Lei wrote: During the stress test, the IO request coroutine has a probability that it can't be awakened when the NBD server is killed. The GDB statck is as follows: (gdb)

Re: [PATCH] hw/block: m25p80: Add support for w25q01jvq

2022-03-08 Thread Cédric Le Goater
On 3/4/22 19:09, Patrick Williams wrote: The w25q01jvq is a 128MB part. Support is being added to the kernel[1] and the two have been tested together. 1. https://lore.kernel.org/lkml/2022022209.23108-1-potin@quantatw.com/ Signed-off-by: Patrick Williams Cc: Potin Lai ---