[PATCH v2 13/17] iotests/migration-permissions: use assertRaises() for qemu_io() negative test

2022-03-24 Thread John Snow
unittests expects you to want to inspect the Exception object, so they leave it defined post-exit.) Signed-off-by: John Snow Reviewed-by: Eric Blake Tested-by: Eric Blake Reviewed-by: Hanna Reitz --- .../qemu-iotests/tests/migration-permissions | 28 +-- 1 file changed, 14

[PATCH v2 08/17] iotests/030: fixup

2022-03-24 Thread John Snow
(Merge into prior patch.) Signed-off-by: John Snow --- tests/qemu-iotests/030 | 85 -- 1 file changed, 49 insertions(+), 36 deletions(-) diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030 index 567bf1da67..3a2de920a3 100755 --- a/tests/qemu

[PATCH v2 07/17] iotests: rebase qemu_io() on top of qemu_tool()

2022-03-24 Thread John Snow
-fleecing migrate-bitmaps-postcopy-test migrate-bitmaps-test migrate-during-backup migration-permissions Test that use qemu_io_log(): 242 245 255 274 303 307 nbd-reconnect-on-open Signed-off-by: John Snow --- Note: This breaks several tests at this point. I'll be fixing each broken test one

[PATCH v2 10/17] iotests/205: fixup

2022-03-24 Thread John Snow
(Merge into prior patch.) Signed-off-by: John Snow --- tests/qemu-iotests/205 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/205 b/tests/qemu-iotests/205 index c0e107328f..15f798288a 100755 --- a/tests/qemu-iotests/205 +++ b/tests/qemu-iotests/205

[PATCH v2 02/17] iotests/163: Fix broken qemu-io invocation

2022-03-24 Thread John Snow
actually enforces that this command completes successfully, but a forthcoming commit in this series will enforce that qemu_io() must have a zero status code.) Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Hanna Reitz --- tests/qemu-iotests/163 | 5 + 1 file changed, 1

[PATCH v2 05/17] iotests/040: Fix TestCommitWithFilters test

2022-03-24 Thread John Snow
groups) will succeed. [Patch entirely written by Hanna but I don't have her S-o-B] [My commit message is probably also garbage, sorry] [Feel free to suggest a better one] [I hope your day is going well] Signed-off-by: John Snow Signed-off-by: John Snow --- tests/qemu-iotests/040 | 6 -- 1

[PATCH v2 01/17] iotests: replace calls to log(qemu_io(...)) with qemu_io_log()

2022-03-24 Thread John Snow
This makes these callsites a little simpler, but the real motivation is a forthcoming commit will change the return type of qemu_io(), so removing users of the return value now is helpful. Signed-off-by: John Snow Reviewed-by: Eric Blake --- tests/qemu-iotests/242 | 6 +++--- tests/qemu

[PATCH v2 06/17] iotests: create generic qemu_tool() function

2022-03-24 Thread John Snow
reimplement qemu_img() in terms of qemu_tool() in preparation for doing the same with qemu_io(). Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Hanna Reitz --- tests/qemu-iotests/iotests.py | 32 +--- 1 file changed, 21 insertions(+), 11 deletions

[PATCH v2 00/17] iotests: add enhanced debugging info to qemu-io failures

2022-03-24 Thread John Snow
a regular CalledProcessError. I think I'd have to extend Popen and add buffering. I think that's something for later. Anyway, that's my story. How are you today? I hope you are well.) John Snow (17): iotests: replace calls to log(qemu_io(...)) with qemu_io_log() iotests/163: Fix broken qemu-io

[PATCH v2 04/17] iotests/040: Don't check image pattern on zero-length image

2022-03-24 Thread John Snow
qemu-io fails on read/write beyond end-of-file on raw images, so skip these invocations when running the zero-length image tests. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Hanna Reitz --- tests/qemu-iotests/040 | 14 -- 1 file changed, 12 insertions(+), 2

[PATCH v2 03/17] iotests: Don't check qemu_io() output for specific error strings

2022-03-24 Thread John Snow
a status code of zero will be a more robust way to guard against failure. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Hanna Reitz --- tests/qemu-iotests/040 | 33 - tests/qemu-iotests/056 | 2 +- 2 files changed, 17 insertions(+), 18 deletions

Re: [PATCH 01/10] python/aqmp: add explicit GPLv2 license to legacy.py

2022-03-24 Thread John Snow
On Thu, Mar 24, 2022 at 11:25 AM Daniel P. Berrangé wrote: > > On Thu, Mar 24, 2022 at 11:03:12AM -0400, John Snow wrote: > > On Thu, Mar 24, 2022, 5:03 AM Daniel P. Berrangé > > wrote: > > > > > On Thu, Mar 24, 2022 at 09:00:05AM +, Daniel P. Berrangé wrot

Re: [PATCH 01/10] python/aqmp: add explicit GPLv2 license to legacy.py

2022-03-24 Thread John Snow
On Thu, Mar 24, 2022, 5:03 AM Daniel P. Berrangé wrote: > On Thu, Mar 24, 2022 at 09:00:05AM +, Daniel P. Berrangé wrote: > > On Wed, Mar 23, 2022 at 05:47:48PM -0400, John Snow wrote: > > > On Mon, Mar 21, 2022 at 5:08 PM John Snow wrote: > > > > > >

Re: [PATCH 01/10] python/aqmp: add explicit GPLv2 license to legacy.py

2022-03-23 Thread John Snow
On Mon, Mar 21, 2022 at 5:08 PM John Snow wrote: > > The legacy.py module is heavily based on the QMP module by Luiz > Capitulino (et al) which is licensed as explicit GPLv2-only. The async > QMP package is currently licensed similarly, but I intend to relicense > the async pack

Re: [PATCH 09/10] python: rename qemu.aqmp to qemu.qmp

2022-03-23 Thread John Snow
On Wed, Mar 23, 2022 at 2:20 PM Hanna Reitz wrote: > > On 21.03.22 22:08, John Snow wrote: > > Now that we are fully switched over to the new QMP library, move it back > > over the old namespace. This is being done primarily so that we may > > upload this package simp

Re: [PATCH 06/10] python/aqmp: copy qmp docstrings to qemu.aqmp.legacy

2022-03-23 Thread John Snow
On Wed, Mar 23, 2022 at 2:24 PM Hanna Reitz wrote: > > On 21.03.22 22:08, John Snow wrote: > > Copy the docstrings out of qemu.qmp, adjusting them as necessary to > > more accurately reflect the current state of this class. > > > > (Licensing: This is copying and

Re: [PATCH-for-6.2 0/2] hw/block/fdc: Fix CVE-2021-3507

2022-03-22 Thread John Snow
t;> Series > >>>>> Acked-by: Jon Maloy > >>>> > >>>> Philippe, > >>>> I hear from other sources that you earlier have qualified this one as > >>>> "incomplete". > >>>> I am of course aware that

Re: [PATCH v2 1/3] qapi: rename BlockDirtyBitmapMergeSource to BlockDirtyBitmapOrStr

2022-03-22 Thread John Snow
core.json | 6 +++--- > > qemu-img.c | 8 > > 4 files changed, 11 insertions(+), 11 deletions(-) > > The type name does not affect QAPI interface stability, so this is safe. > > Reviewed-by: Eric Blake Sounds right. Acked-by: John Snow

Re: [PATCH 13/15] iotests: remove qemu_io_pipe_and_status()

2022-03-22 Thread John Snow
On Tue, Mar 22, 2022 at 12:39 PM Hanna Reitz wrote: > > On 18.03.22 21:36, John Snow wrote: > > I know we just added it, sorry. This is done in favor of qemu_io() which > > *also* returns the console output and status, but with more robust error > > handling on fai

[PATCH] iotests: update test owner contact information

2022-03-22 Thread John Snow
Quite a few of these tests have stale contact information. This patch updates the stale ones that I happen to be aware of at the moment. Signed-off-by: John Snow --- tests/qemu-iotests/025 | 2 +- tests/qemu-iotests/027 | 2 +- tests/qemu-iotests/028 | 2 +- tests/qemu-iotests/036 | 2 +- tests

Re: [PATCH 14/15] iotests: remove qemu_io_silent() and qemu_io_silent_check().

2022-03-22 Thread John Snow
On Tue, Mar 22, 2022 at 1:00 PM Hanna Reitz wrote: > > On 18.03.22 21:36, John Snow wrote: > > Like qemu-img, qemu-io returning 0 should be the norm and not the > > exception. Remove all calls to qemu_io_silent that just assert the > > return code is zero (That's every la

Re: [PATCH 12/15] iotests/migration-permissions: use assertRaises() for qemu_io() negative test

2022-03-22 Thread John Snow
On Tue, Mar 22, 2022 at 12:37 PM Hanna Reitz wrote: > > On 18.03.22 21:36, John Snow wrote: > > Modify this test to use assertRaises for its negative testing of > > qemu_io. If the exception raised does not match the one we tell it to > > expect, we get *that* except

Re: [PATCH 10/15] iotests/245: fixup

2022-03-22 Thread John Snow
On Fri, Mar 18, 2022 at 4:37 PM John Snow wrote: > > (Merge with prior patch.) > > Signed-off-by: John Snow > --- > tests/qemu-iotests/242 | 2 +- ^ Oh, there's the stray import changes that needed to be folded into patch #1. Fixed for v2.

Re: [PATCH 10/15] iotests/245: fixup

2022-03-22 Thread John Snow
On Tue, Mar 22, 2022, 12:30 PM Hanna Reitz wrote: > On 18.03.22 21:36, John Snow wrote: > > (Merge with prior patch.) > > > > Signed-off-by: John Snow > > --- > > tests/qemu-iotests/242 | 2 +- > > tests/qemu-iotests/245 | 2 +- > >

Re: [PATCH 06/15] iotests: rebase qemu_io() on top of qemu_tool()

2022-03-22 Thread John Snow
On Tue, Mar 22, 2022, 11:04 AM Hanna Reitz wrote: > On 18.03.22 21:36, John Snow wrote: > > Rework qemu_io() to be analogous to qemu_img(); a function that requires > > a return code of zero by default unless disabled explicitly. > > > > Tests that use qemu_io(): >

Re: [PATCH 05/15] iotests: create generic qemu_tool() function

2022-03-22 Thread John Snow
On Tue, Mar 22, 2022, 10:49 AM Hanna Reitz wrote: > On 18.03.22 21:36, John Snow wrote: > > reimplement qemu_img() in terms of qemu_tool() in preparation for doing > > the same with qemu_io(). > > > > Signed-off-by: John Snow > > --- > &g

Re: [PATCH 04/15] iotests/040: Don't check image pattern on zero-length image

2022-03-22 Thread John Snow
On Tue, Mar 22, 2022, 10:22 AM Hanna Reitz wrote: > On 18.03.22 21:36, John Snow wrote: > > qemu-io fails on read/write with zero-length raw images, so skip these > > when running the zero-length image tests. > > > > Signed-off-by: John Snow > > ---

[PATCH 10/10] python: rename 'aqmp-tui' to 'qmp-tui'

2022-03-21 Thread John Snow
This is the last vestige of the "aqmp" moniker surviving in the tree; remove it. Signed-off-by: John Snow Reviewed-by: Beraldo Leal --- python/qemu/qmp/{aqmp_tui.py => qmp_tui.py} | 12 ++-- python/setup.cfg| 6 +++--- 2 files changed, 9 ins

[PATCH 09/10] python: rename qemu.aqmp to qemu.qmp

2022-03-21 Thread John Snow
ade-off is increased confusion inside the QEMU developer tree. Sorry! Note: the 'private' member "_aqmp" in legacy.py also changes to "_qmp"; not out of necessity, but just to remove any traces of the "aqmp" name. Signed-off-by: John Snow Reviewed-by: Beraldo Leal -

[PATCH 07/10] python: remove the old QMP package

2022-03-21 Thread John Snow
Thank you for your service! Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Beraldo Leal --- python/PACKAGE.rst | 4 +- python/README.rst | 2 +- python/qemu/qmp/README.rst | 9 - python/qemu/qmp/__init__.py | 396

Re: [PATCH v2 0/4] iotests: finalize switch to async QMP

2022-03-21 Thread John Snow
On Mon, Mar 21, 2022 at 4:33 PM John Snow wrote: > > GitLab: https://gitlab.com/jsnow/qemu/-/commits/python-qmp-legacy-switch-pt1b > CI: https://gitlab.com/jsnow/qemu/-/pipelines/495951187 > > This series isolates the iotests-centric changes required to switch to > the new QMP l

[PATCH 05/10] python/aqmp: fully separate from qmp.QEMUMonitorProtocol

2022-03-21 Thread John Snow
After this patch, qemu.aqmp.legacy.QEMUMonitorProtocol no longer inherits from qemu.qmp.QEMUMonitorProtocol. To do this, several inherited methods need to be explicitly re-defined. (Licensing: This is copying and modifying GPLv2-only code into a GPLv2-only file.) Signed-off-by: John Snow

[PATCH 04/10] python/aqmp: take QMPBadPortError and parse_address from qemu.qmp

2022-03-21 Thread John Snow
Shift these definitions over from the qmp package to the async qmp package. (Licensing: this is a lateral move, from GPLv2 (only) to GPLv2 (only)) Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Beraldo Leal --- python/qemu/aqmp/aqmp_tui.py | 3 +-- python

[PATCH 02/10] python/aqmp: relicense as GPLv2+

2022-03-21 Thread John Snow
project. Signed-off-by: John Snow --- python/qemu/aqmp/__init__.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/python/qemu/aqmp/__init__.py b/python/qemu/aqmp/__init__.py index 4c22c38079..2e4478ac05 100644 --- a/python/qemu/aqmp/__init__.py +++ b/python/qemu/aqmp

[PATCH 08/10] python: re-enable pylint duplicate-code warnings

2022-03-21 Thread John Snow
With the old library gone, there's nothing duplicated in the tree, so the warning suppression can be removed. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Beraldo Leal --- python/setup.cfg | 1 - 1 file changed, 1 deletion(-) diff --git a/python/setup.cfg b

[PATCH 03/10] python: temporarily silence pylint duplicate-code warnings

2022-03-21 Thread John Snow
The next several commits copy some code from qemu.qmp to qemu.aqmp, then delete qemu.qmp. In the interim, to prevent test failures, the duplicate code detection needs to be silenced to prevent bisect problems with CI testing. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy

[PATCH 06/10] python/aqmp: copy qmp docstrings to qemu.aqmp.legacy

2022-03-21 Thread John Snow
Copy the docstrings out of qemu.qmp, adjusting them as necessary to more accurately reflect the current state of this class. (Licensing: This is copying and modifying GPLv2-only licensed docstrings into a GPLv2-only file.) Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy

[PATCH 01/10] python/aqmp: add explicit GPLv2 license to legacy.py

2022-03-21 Thread John Snow
the license on legacy.py explicit. Signed-off-by: John Snow --- python/qemu/aqmp/legacy.py | 11 +++ 1 file changed, 11 insertions(+) diff --git a/python/qemu/aqmp/legacy.py b/python/qemu/aqmp/legacy.py index 46026e9fdc..f86cb29804 100644 --- a/python/qemu/aqmp/legacy.py +++ b/python/qemu/aqmp

[PATCH 00/10] Python: Remove synchronous QMP library

2022-03-21 Thread John Snow
elease cycles now, it's time. John Snow (10): python/aqmp: add explicit GPLv2 license to legacy.py python/aqmp: relicense as GPLv2+ python: temporarily silence pylint duplicate-code warnings python/aqmp: take QMPBadPortError and parse_address from qemu.qmp python/aqmp: fully separat

[PATCH v5 18/18] iotests: make qemu_img_log and img_info_log raise on error

2022-03-21 Thread John Snow
-off-by: John Snow --- tests/qemu-iotests/242| 2 +- tests/qemu-iotests/266| 2 +- tests/qemu-iotests/iotests.py | 8 +--- 3 files changed, 7 insertions(+), 5 deletions(-) diff --git a/tests/qemu-iotests/242 b/tests/qemu-iotests/242 index 547bf382e3..b3afd36d72 100755

[PATCH v2 4/4] iotests: switch to AQMP

2022-03-21 Thread John Snow
iotests is already using async QMP, but to finalize the switchover we only need to update any remaining import paths to rely solely on the new library instead. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Beraldo Leal Acked-by: Hanna Reitz --- tests/qemu

[PATCH v2 3/4] iotests/mirror-top-perms: switch to AQMP

2022-03-21 Thread John Snow
with. Ultimately, I decided it's fine to just suppress the logger temporarily.) Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Acked-by: Hanna Reitz --- tests/qemu-iotests/tests/mirror-top-perms | 9 +++-- 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/tests/qemu

[PATCH v2 0/4] iotests: finalize switch to async QMP

2022-03-21 Thread John Snow
for the purposes of aggregating tags before pulling it into my Python branch. John Snow (4): python/machine: permanently switch to AQMP scripts/bench-block-job: switch to AQMP iotests/mirror-top-perms: switch to AQMP iotests: switch to AQMP python/qemu/machine/machine.py| 18

[PATCH v2 2/4] scripts/bench-block-job: switch to AQMP

2022-03-21 Thread John Snow
For this commit, we only need to remove accommodations for the synchronous QMP library. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Beraldo Leal Acked-by: Hanna Reitz --- scripts/simplebench/bench_block_job.py | 3 +-- 1 file changed, 1 insertion(+), 2

[PATCH v2 1/4] python/machine: permanently switch to AQMP

2022-03-21 Thread John Snow
Remove the QEMU_PYTHON_LEGACY_QMP environment variable, making the switch from sync qmp to async qmp permanent. Update exceptions and import paths as necessary. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Beraldo Leal Acked-by: Hanna Reitz --- python/qemu

[PATCH v5 14/18] iotests: remove remaining calls to qemu_img_pipe()

2022-03-21 Thread John Snow
iotests.py and will be removed in subsequent patches. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/065 | 4 ++-- tests/qemu-iotests/237 | 3 +-- tests/qemu-iotests/237.out | 3 --- tests/qemu-iotests/296 | 12 ++-- 4 files changed, 9 insertions(+), 13 deletion

[PATCH v5 12/18] iotests: replace unchecked calls to qemu_img_pipe()

2022-03-21 Thread John Snow
, and 303. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/194 | 4 ++-- tests/qemu-iotests/202 | 4 ++-- tests/qemu-iotests/203 | 4 ++-- tests/qemu-iotests/234 | 4 ++-- tests/qemu-iotests/262 | 2 +- tests/qemu-iotests/303 | 2 +- 6 files changed, 10 insertions(+), 10

[PATCH v5 17/18] iotests: remove qemu_img_pipe_and_status()

2022-03-21 Thread John Snow
to enforce a successful exit code by default even for the logged functions. Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 26 +++--- 1 file changed, 7 insertions(+), 19 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index

[PATCH v5 15/18] iotests: use qemu_img() in has_working_luks()

2022-03-21 Thread John Snow
Admittedly a mostly lateral move, but qemu_img() is essentially the replacement for qemu_img_pipe_and_status(). It will give slightly better diagnostics on crash. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/iotests.py | 18 +- 1 file changed, 9

[PATCH v5 07/18] iotests: use qemu_img_json() when applicable

2022-03-21 Thread John Snow
qemu_img_json() gives better diagnostic information on failure. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/iotests.py | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index

[PATCH v5 16/18] iotests: replace qemu_img_log('create', ...) calls

2022-03-21 Thread John Snow
where appropriate. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/255 | 8 tests/qemu-iotests/255.out | 4 tests/qemu-iotests/274 | 17 - tests/qemu-iotests/274.out | 29 - tests/qemu-iotests/280 | 2

[PATCH v5 13/18] iotests/149: Remove qemu_img_pipe() call

2022-03-21 Thread John Snow
to inspect the actual output to see if we want to retroactively skip the test due to missing cipher support. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/149 | 7 +-- tests/qemu-iotests/149.out | 21 - 2 files changed, 5 insertions(+), 23

[PATCH v5 03/18] iotests: Remove explicit checks for qemu_img() == 0

2022-03-21 Thread John Snow
qemu_img() returning zero ought to be the rule, not the exception. Remove all explicit checks against the condition in preparation for making non-zero returns an Exception. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Hanna Reitz --- tests/qemu-iotests/163

[PATCH v5 10/18] iotests: add qemu_img_map() function

2022-03-21 Thread John Snow
for iotest 211 changes, because logging JSON after it was deserialized by Python behaves a little differently than logging the raw JSON document string itself. (iotests.log() sorts the keys for Python 3.6 support.) Signed-off-by: John Snow Reviewed-by: Eric Blake --- tests/qemu-iotests/041

[PATCH v5 11/18] iotests: change supports_quorum to use qemu_img

2022-03-21 Thread John Snow
u_img_pipe() has to be removed.) Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/iotests.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index b3e793f78a..aaf4da8be4 100644 --- a/

[PATCH v5 08/18] iotests: add qemu_img_info()

2022-03-21 Thread John Snow
Add qemu_img_info() by analogy with qemu_img_measure() and qemu_img_check(). Modify image_size() to use this function instead to take advantage of the better diagnostic information on failure provided (ultimately) by qemu_img(). Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu

[PATCH v5 04/18] iotests: make qemu_img raise on non-zero rc by default

2022-03-21 Thread John Snow
-img. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Hanna Reitz --- tests/qemu-iotests/257| 8 +++-- tests/qemu-iotests/iotests.py | 57 ++- 2 files changed, 55 insertions(+), 10 deletions(-) diff --git a/tests/qemu-iotests/257 b/tests

[PATCH v5 09/18] iotests/remove-bitmap-from-backing: use qemu_img_info()

2022-03-21 Thread John Snow
This removes two more usages of qemu_img_pipe() and replaces them with calls to qemu_img(), which provides better diagnostic information on failure. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/tests/remove-bitmap-from-backing | 6 +++--- 1 file changed, 3

[PATCH v5 06/18] iotests: add qemu_img_json()

2022-03-21 Thread John Snow
by qemu_img() is re-raised. In the event that the return code is zero but we can't parse valid JSON, allow the JSON deserialization error to be raised. Signed-off-by: John Snow Reviewed-by: Eric Blake --- tests/qemu-iotests/iotests.py | 32 1 file changed, 32

[PATCH v5 00/18] iotests: add enhanced debugging info to qemu-img failures

2022-03-21 Thread John Snow
or *all* qemu-img calls, logged or not. John Snow (18): python/utils: add add_visual_margin() text decoration utility python/utils: add VerboseProcessError iotests: Remove explicit checks for qemu_img() == 0 iotests: make qemu_img raise on non-zero rc by default iotests: fortify compare

[PATCH v5 02/18] python/utils: add VerboseProcessError

2022-03-21 Thread John Snow
so that it's easy to spot in a sea of traceback information. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Hanna Reitz --- python/qemu/utils/__init__.py | 39 +++ 1 file changed, 39 insertions(+) diff --git a/python/qemu/utils/__init__.py b

[PATCH v5 05/18] iotests: fortify compare_images() against crashes

2022-03-21 Thread John Snow
(perhaps to test how qemu_img compare behaves on malformed images, for instance), it is still possible to catch the exception in the test and deal with that circumstance manually. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Hanna Reitz --- tests/qemu-iotests/iotests.py | 21

[PATCH v5 01/18] python/utils: add add_visual_margin() text decoration utility

2022-03-21 Thread John Snow
label may be given, and any of the individual glyphs used to draw the ┃ box may be replaced or specified as well. ┗━━━ Signed-off-by: John Snow Reviewed-by: Eric Blake Acked-by: Hanna Reitz --- python/qemu/uti

Re: [PATCH 15/15] iotests: make qemu_io_log() check return codes by default

2022-03-21 Thread John Snow
On Mon, Mar 21, 2022 at 2:22 PM Eric Blake wrote: > > On Fri, Mar 18, 2022 at 04:36:55PM -0400, John Snow wrote: > > Just like qemu_img_log(), upgrade qemu_io_log() to enforce a return code > > of zero by default. > > > > Affected tests: 242 245 255 274

Re: [PATCH 14/15] iotests: remove qemu_io_silent() and qemu_io_silent_check().

2022-03-21 Thread John Snow
On Mon, Mar 21, 2022 at 2:16 PM Eric Blake wrote: > > On Fri, Mar 18, 2022 at 04:36:54PM -0400, John Snow wrote: > > Like qemu-img, qemu-io returning 0 should be the norm and not the > > exception. Remove all calls to qemu_io_silent that just assert the > > return code is

Re: [PATCH 06/15] iotests: rebase qemu_io() on top of qemu_tool()

2022-03-21 Thread John Snow
On Mon, Mar 21, 2022, 11:29 AM Eric Blake wrote: > On Fri, Mar 18, 2022 at 04:36:46PM -0400, John Snow wrote: > > Rework qemu_io() to be analogous to qemu_img(); a function that requires > > a return code of zero by default unless disabled explicitly. > > > > Tests

Re: [PATCH v4 00/18] iotests: add enhanced debugging info to qemu-img failures

2022-03-21 Thread John Snow
On Mon, Mar 21, 2022, 11:50 AM Hanna Reitz wrote: > On 21.03.22 14:14, Hanna Reitz wrote: > > On 18.03.22 22:14, John Snow wrote: > >> On Fri, Mar 18, 2022 at 9:36 AM Hanna Reitz wrote: > >>> On 18.03.22 00:49, John Snow wrote: > >>>> Hiya! &

Re: [PATCH 05/15] iotests: create generic qemu_tool() function

2022-03-21 Thread John Snow
On Mon, Mar 21, 2022, 11:13 AM Eric Blake wrote: > On Fri, Mar 18, 2022 at 04:36:45PM -0400, John Snow wrote: > > reimplement qemu_img() in terms of qemu_tool() in preparation for doing > > the same with qemu_io(). > > > > Signed-off-by: John Snow > > --- > &

Re: [PATCH v4 10/18] iotests: add qemu_img_map() function

2022-03-21 Thread John Snow
On Mon, Mar 21, 2022, 10:24 AM Eric Blake wrote: > On Thu, Mar 17, 2022 at 07:49:29PM -0400, John Snow wrote: > > Add a qemu_img_map() function by analogy with qemu_img_measure(), > > qemu_img_check(), and qemu_img_info() that all return JSON information. > > > > Rep

Re: [PATCH 2/2] iotests/207: Filter host fingerprint

2022-03-18 Thread John Snow
baf5e4da4 >("block: print the server key type and fingerprint on failure") > Reported-by: John Snow > Signed-off-by: Hanna Reitz > --- > tests/qemu-iotests/207 | 7 ++- > tests/qemu-iotests/207.out | 6 +++--- > 2 files changed, 9 insertions(+), 4 deletions

Re: [PATCH 1/2] iotests.py: Filters for VM.run_job()

2022-03-18 Thread John Snow
y: Hanna Reitz Looks fine enough to me for now. Reviewed-by: John Snow > --- > tests/qemu-iotests/iotests.py | 26 -- > 1 file changed, 16 insertions(+), 10 deletions(-) > > diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py &

Re: [PATCH v4 00/18] iotests: add enhanced debugging info to qemu-img failures

2022-03-18 Thread John Snow
On Fri, Mar 18, 2022 at 9:36 AM Hanna Reitz wrote: > > On 18.03.22 00:49, John Snow wrote: > > Hiya! > > > > This series effectively replaces qemu_img_pipe_and_status() with a > > rewritten function named qemu_img() that raises an exception on non-zero > >

[PATCH 14/15] iotests: remove qemu_io_silent() and qemu_io_silent_check().

2022-03-18 Thread John Snow
Like qemu-img, qemu-io returning 0 should be the norm and not the exception. Remove all calls to qemu_io_silent that just assert the return code is zero (That's every last call, as it turns out), and replace them with a normal qemu_io() call. Signed-off-by: John Snow --- tests/qemu-iotests/216

[PATCH 05/15] iotests: create generic qemu_tool() function

2022-03-18 Thread John Snow
reimplement qemu_img() in terms of qemu_tool() in preparation for doing the same with qemu_io(). Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 37 +++ 1 file changed, 24 insertions(+), 13 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b

[PATCH 10/15] iotests/245: fixup

2022-03-18 Thread John Snow
(Merge with prior patch.) Signed-off-by: John Snow --- tests/qemu-iotests/242 | 2 +- tests/qemu-iotests/245 | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/242 b/tests/qemu-iotests/242 index 4b7ec16af6..ecc851582a 100755 --- a/tests/qemu-iotests/242

[PATCH 13/15] iotests: remove qemu_io_pipe_and_status()

2022-03-18 Thread John Snow
I know we just added it, sorry. This is done in favor of qemu_io() which *also* returns the console output and status, but with more robust error handling on failure. Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 3 --- tests/qemu-iotests/tests/image-fleecing | 12

[PATCH 15/15] iotests: make qemu_io_log() check return codes by default

2022-03-18 Thread John Snow
Just like qemu_img_log(), upgrade qemu_io_log() to enforce a return code of zero by default. Affected tests: 242 245 255 274 303 307 nbd-reconnect-on-open Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 5 +++-- tests/qemu-iotests/tests/nbd-reconnect-on-open | 2

[PATCH 06/15] iotests: rebase qemu_io() on top of qemu_tool()

2022-03-18 Thread John Snow
-fleecing migrate-bitmaps-postcopy-test migrate-bitmaps-test migrate-during-backup migration-permissions Test that use qemu_io_log(): 242 245 255 274 303 307 nbd-reconnect-on-open Signed-off-by: John Snow --- Note: This breaks several tests at this point. I'll be fixing each broken test one

[PATCH 09/15] iotests/205: fixup

2022-03-18 Thread John Snow
(Merge into prior patch.) Signed-off-by: John Snow --- tests/qemu-iotests/205 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/205 b/tests/qemu-iotests/205 index c0e107328f..15f798288a 100755 --- a/tests/qemu-iotests/205 +++ b/tests/qemu-iotests/205

[PATCH 08/15] iotests/149: fixup

2022-03-18 Thread John Snow
(Merge into prior patch.) Notes: I don't quite like this change, but I'm at a loss for what would be cleaner. This is a funky test. Signed-off-by: John Snow --- tests/qemu-iotests/149 | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/149 b/tests/qemu

[PATCH 11/15] iotests/migration-permissions: fixup

2022-03-18 Thread John Snow
(Merge into prior commit) Note, this is a quick hack band-aid, but a follow-up patch spends the time to refactor it a bit. This is just the quick stop-gap to prevent bisection failures. Signed-off-by: John Snow --- tests/qemu-iotests/tests/migration-permissions | 4 ++-- 1 file changed, 2

[PATCH 00/15] iotests: add enhanced debugging info to qemu-io failures

2022-03-18 Thread John Snow
ot open backing file: Throttle ┃ group 'tg' does not exist ┗━ It looks like we start with the img chain 3->2->1->0, then we commit 2 down into 1, but checking '3' fails somewhere in the backing chain. Maybe a real bu

[PATCH 04/15] iotests/040: Don't check image pattern on zero-length image

2022-03-18 Thread John Snow
qemu-io fails on read/write with zero-length raw images, so skip these when running the zero-length image tests. Signed-off-by: John Snow --- tests/qemu-iotests/040 | 14 -- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests

[PATCH 03/15] iotests: Don't check qemu_io() output for specific error strings

2022-03-18 Thread John Snow
a status code of zero will be a more robust way to guard against failure. Signed-off-by: John Snow --- tests/qemu-iotests/040 | 33 - tests/qemu-iotests/056 | 2 +- 2 files changed, 17 insertions(+), 18 deletions(-) diff --git a/tests/qemu-iotests/040 b/tests/qemu

[PATCH 07/15] iotests/030: fixup

2022-03-18 Thread John Snow
(Merge into prior patch.) Signed-off-by: John Snow --- tests/qemu-iotests/030 | 85 -- 1 file changed, 49 insertions(+), 36 deletions(-) diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030 index 567bf1da67..3a2de920a3 100755 --- a/tests/qemu

[PATCH 02/15] iotests/163: Fix broken qemu-io invocation

2022-03-18 Thread John Snow
actually enforces that this command completes successfully, but a forthcoming commit in this series will enforce that qemu_io() must have a zero status code.) Signed-off-by: John Snow --- tests/qemu-iotests/163 | 5 + 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/tests/qemu

[PATCH 12/15] iotests/migration-permissions: use assertRaises() for qemu_io() negative test

2022-03-18 Thread John Snow
the CalledProcessError exception but the output is not what we expect, we re-raise the original CalledProcessError. Tidy. Signed-off-by: John Snow --- .../qemu-iotests/tests/migration-permissions | 28 +-- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/tests/qemu

[PATCH 01/15] iotests: replace calls to log(qemu_io(...)) with qemu_io_log()

2022-03-18 Thread John Snow
This makes these callsites a little simpler, but the real motivation is a forthcoming commit will change the return type of qemu_io(), so removing users of the return value now is helpful. Signed-off-by: John Snow --- tests/qemu-iotests/242 | 2 +- tests/qemu-iotests/255 | 4 +--- tests/qemu

Re: [PATCH 0/4] iotests: finalize switch to async QMP

2022-03-18 Thread John Snow
On Fri, Mar 18, 2022 at 12:32 PM Hanna Reitz wrote: > > On 08.02.22 20:52, John Snow wrote: > > Squeak Squeak... > > > > ...Any objections to me staging this? > > > > (This patchset removes the accommodations in iotests for allowing > > either library t

Re: [PATCH v4 00/18] iotests: add enhanced debugging info to qemu-img failures

2022-03-18 Thread John Snow
On Fri, Mar 18, 2022, 9:36 AM Hanna Reitz wrote: > On 18.03.22 00:49, John Snow wrote: > > Hiya! > > > > This series effectively replaces qemu_img_pipe_and_status() with a > > rewritten function named qemu_img() that raises an exception on non-zero > > r

[PATCH v4 04/18] iotests: make qemu_img raise on non-zero rc by default

2022-03-17 Thread John Snow
-img. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Hanna Reitz --- tests/qemu-iotests/257| 8 +++-- tests/qemu-iotests/iotests.py | 56 ++- 2 files changed, 54 insertions(+), 10 deletions(-) diff --git a/tests/qemu-iotests/257 b/tests

[PATCH v4 10/18] iotests: add qemu_img_map() function

2022-03-17 Thread John Snow
for iotest 211 changes, because logging JSON after it was deserialized by Python behaves a little differently than logging the raw JSON document string itself. (iotests.log() sorts the keys for Python 3.6 support.) Signed-off-by: John Snow --- tests/qemu-iotests/041 | 5

[PATCH v4 07/18] iotests: use qemu_img_json() when applicable

2022-03-17 Thread John Snow
qemu_img_json() gives better diagnostic information on failure. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/iotests.py | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index

[PATCH v4 14/18] iotests: remove remaining calls to qemu_img_pipe()

2022-03-17 Thread John Snow
iotests.py and will be removed in subsequent patches. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/065 | 4 ++-- tests/qemu-iotests/237 | 3 +-- tests/qemu-iotests/237.out | 3 --- tests/qemu-iotests/296 | 12 ++-- 4 files changed, 9 insertions(+), 13 deletion

[PATCH v4 18/18] iotests: make qemu_img_log and img_info_log raise on error

2022-03-17 Thread John Snow
-off-by: John Snow --- tests/qemu-iotests/242| 2 +- tests/qemu-iotests/266| 2 +- tests/qemu-iotests/iotests.py | 8 +--- 3 files changed, 7 insertions(+), 5 deletions(-) diff --git a/tests/qemu-iotests/242 b/tests/qemu-iotests/242 index 547bf382e3..b3afd36d72 100755

[PATCH v4 16/18] iotests: replace qemu_img_log('create', ...) calls

2022-03-17 Thread John Snow
where appropriate. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/255 | 8 tests/qemu-iotests/255.out | 4 tests/qemu-iotests/274 | 17 - tests/qemu-iotests/274.out | 29 - tests/qemu-iotests/280 | 2

[PATCH v4 09/18] iotests/remove-bitmap-from-backing: use qemu_img_info()

2022-03-17 Thread John Snow
This removes two more usages of qemu_img_pipe() and replaces them with calls to qemu_img(), which provides better diagnostic information on failure. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/tests/remove-bitmap-from-backing | 6 +++--- 1 file changed, 3

[PATCH v4 08/18] iotests: add qemu_img_info()

2022-03-17 Thread John Snow
Add qemu_img_info() by analogy with qemu_img_measure() and qemu_img_check(). Modify image_size() to use this function instead to take advantage of the better diagnostic information on failure provided (ultimately) by qemu_img(). Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu

[PATCH v4 17/18] iotests: remove qemu_img_pipe_and_status()

2022-03-17 Thread John Snow
to enforce a successful exit code by default even for the logged functions. Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 26 +++--- 1 file changed, 7 insertions(+), 19 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index

[PATCH v4 11/18] iotests: change supports_quorum to use qemu_img

2022-03-17 Thread John Snow
u_img_pipe() has to be removed.) Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/iotests.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 58fb1734b5..17065fc176 100644 --- a/

<    1   2   3   4   5   6   7   8   9   10   >