Re: Failing QEMU iotests

2021-11-17 Thread John Snow
On Wed, Nov 17, 2021 at 5:07 AM Thomas Huth wrote: > > Hi! > > I think it has been working fine for me a couple of weeks ago, > but when I now run: > > make check SPEED=slow > > I'm getting a couple of failing iotests... not sure whether > these are known issues already, so I thought I'd

Re: Failing QEMU iotests

2021-11-17 Thread John Snow
On Wed, Nov 17, 2021 at 7:50 AM Thomas Huth wrote: > On 17/11/2021 11.59, Hanna Reitz wrote: > > On 17.11.21 11:07, Thomas Huth wrote: > > >> +++ 297.out.bad > >> @@ -1,2 +1,21 @@ > >> === pylint === > >> +* Module image-fleecing > >> +tests/image-fleecing:34:24: C0326: Exactly one

Re: [PULL 0/5] Python patches

2021-11-17 Thread John Snow
On Wed, Nov 17, 2021 at 4:42 AM Gerd Hoffmann wrote: > Hi, > > > https://gitlab.com/jsnow/qemu.git tags/python-pull-request > > What is the status of the plan to upload this to pypi eventually? > > Thanks for asking! The honest answer is "I'm not exactly sure", but there are a few things to

[PULL 5/5] scripts/device-crash-test: hide tracebacks for QMP connect errors

2021-11-16 Thread John Snow
Generally, the traceback for a connection failure is uninteresting and all we need to know is that the connection attempt failed. Reduce the verbosity in these cases, except when debugging. Signed-off-by: John Snow Reported-by: Thomas Huth Tested-by: Thomas Huth Message-id

[PULL 3/5] scripts/device-crash-test: simplify Exception handling

2021-11-16 Thread John Snow
rruptError and a few other important ones are BaseExceptions, so this does the same thing with less code. Signed-off-by: John Snow Reported-by: Thomas Huth Tested-by: Thomas Huth Message-id: 2021143719.2162525-4-js...@redhat.com Signed-off-by: John Snow --- scripts/device-crash-test | 8 ++

[PULL 1/5] python/aqmp: Fix disconnect during capabilities negotiation

2021-11-16 Thread John Snow
same" exception as the one that caused the initial disconnect, but for now the problem can be avoided by improving the error pathway detection in the exit path. Reported-by: Thomas Huth Signed-off-by: John Snow Tested-by: Thomas Huth Message-id: 2021143719.2162525-2-js...@redhat.com

[PULL 2/5] python/aqmp: fix ConnectError string method

2021-11-16 Thread John Snow
to establish session: EOFError Better! Signed-off-by: John Snow Reported-by: Thomas Huth Tested-by: Thomas Huth Message-id: 2021143719.2162525-3-js...@redhat.com Signed-off-by: John Snow --- python/qemu/aqmp/protocol.py | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff

[PULL 4/5] scripts/device-crash-test: don't emit AQMP connection errors to stdout

2021-11-16 Thread John Snow
These errors are expected, so they shouldn't clog up terminal output. In the event that they're *not* expected, we'll be seeing an awful lot more output concerning the nature of the failure. Reported-by: Thomas Huth Signed-off-by: John Snow Tested-by: Thomas Huth Message-id

[PULL 0/5] Python patches

2021-11-16 Thread John Snow
John Snow (5): python/aqmp: Fix disconnect during capabilities negotiation python/aqmp: fix ConnectError string method scripts/device-crash-test: simplify Exception handling scripts/device-crash-test: don't emit AQMP connection errors to stdout scripts/device-crash-test

Re: [PULL 22/22] python, iotests: replace qmp with aqmp

2021-11-09 Thread John Snow
On Tue, Nov 9, 2021 at 9:07 AM Thomas Huth wrote: > On 01/11/2021 18.30, John Snow wrote: > > Swap out the synchronous QEMUMonitorProtocol from qemu.qmp with the sync > > wrapper from qemu.aqmp instead. > > > > Add an escape hatch in the form

[PULL 22/22] python, iotests: replace qmp with aqmp

2021-11-01 Thread John Snow
work concurrently. Signed-off-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Hanna Reitz Message-id: 20211026175612.4127598-9-js...@redhat.com Signed-off-by: John Snow --- python/qemu/machine/machine.py | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/python/qemu

[PULL 21/22] python/aqmp: Create sync QMP wrapper for iotests

2021-11-01 Thread John Snow
This is a wrapper around the async QMPClient that mimics the old, synchronous QEMUMonitorProtocol class. It is designed to be interchangeable with the old implementation. It does not, however, attempt to mimic Exception compatibility. Signed-off-by: John Snow Acked-by: Hanna Reitz Reviewed

[PULL 16/22] python/machine: Handle QMP errors on close more meticulously

2021-11-01 Thread John Snow
' command. That error will ultimately be bubbled up when attempting to close the QMP connection. The manager class here then is free to discard it -- if it was expected. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Reviewed-by: Kevin Wolf Message-id: 20211026175612.4127598-3-js...@redhat.com

[PULL 15/22] python/machine: remove has_quit argument

2021-11-01 Thread John Snow
ut down. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Reviewed-by: Kevin Wolf Message-id: 20211026175612.4127598-2-js...@redhat.com Signed-off-by: John Snow --- python/qemu/machine/machine.py | 34 +++--- tests/qemu-iotests/040 | 7 +-- tests/qemu-iote

[PULL 13/22] iotests/linters: Add workaround for mypy bug #9852

2021-11-01 Thread John Snow
es that 0.920 may bring with it.) See also: https://github.com/python/mypy/issues/11010 https://github.com/python/mypy/issues/9852 Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-14-js...@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/linters.py | 4

[PULL 09/22] iotests/297: update tool availability checks

2021-11-01 Thread John Snow
As mentioned in 'iotests/297: Don't rely on distro-specific linter binaries', these checks are overly strict. Update them to be in-line with how we actually invoke the linters themselves. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-10-js...@redhat.com

[PULL 20/22] iotests/300: avoid abnormal shutdown race condition

2021-11-01 Thread John Snow
, which will cause an ECONNRESET error to be logged. Waiting for the VM to exit itself avoids the race on shutdown behavior.) Reported-by: Hanna Reitz Signed-off-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Hanna Reitz Message-id: 20211026175612.4127598-7-js...@redhat.com Signed-off-by: John

[PULL 19/22] iotests: Conditionally silence certain AQMP errors

2021-11-01 Thread John Snow
can't amend the iotest output to just always expect the error message, either. Just suppress it for now. More rigorous log filtering can be investigated later if/when it is deemed safe to permanently replace the legacy QMP library.) Signed-off-by: John Snow Reviewed-by: Hanna Reitz Reviewed

[PULL 18/22] iotests: Accommodate async QMP Exception classes

2021-11-01 Thread John Snow
. That'd be these: - QMPError - QMPConnectError - QMPCapabilitiesError - QMPTimeoutError - QMPProtocolError - QMPResponseError - QMPBadPortError Signed-off-by: John Snow Reviewed-by: Hanna Reitz Reviewed-by: Kevin Wolf Message-id: 20211026175612.4127598-5-js...@redhat.com Signed-off-by: John Snow

[PULL 14/22] python: Add iotest linters to test suite

2021-11-01 Thread John Snow
against a wide variety of python versions, which helps to find accidental cross-version python compatibility issues. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-15-js...@redhat.com Signed-off-by: John Snow --- python/tests/iotests-mypy.sh | 4 python

[PULL 07/22] iotests/297: refactor run_[mypy|pylint] as generic execution shim

2021-11-01 Thread John Snow
There's virtually nothing special here anymore; we can combine these into a single, rather generic function. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-8-js...@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/297 | 42

[PULL 05/22] iotests/297: Don't rely on distro-specific linter binaries

2021-11-01 Thread John Snow
test begins -- this check is now "overly strict", but shouldn't cause anything that was already running correctly to start failing. This is addressed by a commit later in this series; 'iotests/297: update tool availability checks'. Signed-off-by: John Snow Reviewed-by: Vladimir Sement

[PULL 12/22] iotests/linters: Add entry point for linting via Python CI

2021-11-01 Thread John Snow
shebang.) Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-13-js...@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/linters.py | 27 +++ 1 file changed, 27 insertions(+) diff --git a/tests/qemu-iotests/linters.py b/tests

[PULL 17/22] python/aqmp: Remove scary message

2021-11-01 Thread John Snow
The scary message interferes with the iotests output. Coincidentally, if iotests works by removing this, then it's good evidence that we don't really need to scare people away from using it. Signed-off-by: John Snow Acked-by: Hanna Reitz Reviewed-by: Kevin Wolf Reviewed-by: Hanna Reitz

[PULL 11/22] iotests: split linters.py out from 297

2021-11-01 Thread John Snow
configurations both would be ... a hassle. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-12-js...@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/297| 72 + tests/qemu-iotests/linters.py | 76

[PULL 10/22] iotests/297: split test into sub-cases

2021-11-01 Thread John Snow
Take iotest 297's main() test function and split it into two sub-cases that can be skipped individually. We can also drop custom environment setup from the pylint test as it isn't needed. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-11-js...@redhat.com

[PULL 08/22] iotests/297: Change run_linter() to raise an exception on failure

2021-11-01 Thread John Snow
/format that information. (Also, "suppress_output" is now a lot more accurate of a parameter name.) Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-9-js...@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/297 | 24 ++

[PULL 04/22] iotests/297: Create main() function

2021-11-01 Thread John Snow
Instead of running "run_linters" directly, create a main() function that will be responsible for environment setup, leaving run_linters() responsible only for execution of the linters. (That environment setup will be moved over in forthcoming commits.) Signed-off-by: John Snow

[PULL 06/22] iotests/297: Split run_linters apart into run_pylint and run_mypy

2021-11-01 Thread John Snow
Move environment setup into main(), and split the actual linter execution into run_pylint and run_mypy, respectively. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-7-js...@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/297 | 38

[PULL 03/22] iotests/297: Add get_files() function

2021-11-01 Thread John Snow
Split out file discovery into its own method to begin separating out configuration/setup and test execution. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-4-js...@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/297 | 9 +++-- 1 file

[PULL 02/22] iotests/297: Split mypy configuration out into mypy.ini

2021-11-01 Thread John Snow
More separation of code and configuration. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-3-js...@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/297 | 14 +- tests/qemu-iotests/mypy.ini | 12 2 files changed, 13

[PULL 01/22] iotests/297: Move pylint config into pylintrc

2021-11-01 Thread John Snow
Move --score=n and --notes=XXX,FIXME into pylintrc. This pulls configuration out of code, which I think is probably a good thing in general. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-2-js...@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests

[PULL 00/22] Python patches

2021-11-01 Thread John Snow
for you to fetch changes up to 76cd358671e6b8e7c435ec65b1c44200254514a9: python, iotests: replace qmp with aqmp (2021-11-01 11:54:59 -0400) Pull request John Snow (22

Re: [PATCH v2 11/15] iotests: split linters.py out from 297

2021-10-28 Thread John Snow
On Thu, Oct 28, 2021, 6:34 AM Hanna Reitz wrote: > On 26.10.21 20:30, John Snow wrote: > > > > > > On Tue, Oct 26, 2021 at 6:51 AM Hanna Reitz wrote: > > > > On 19.10.21 16:49, John Snow wrote: > > > Now, 297 is just the iotests-sp

Re: [PATCH v5 2/8] python/machine: Handle QMP errors on close more meticulously

2021-10-28 Thread John Snow
On Thu, Oct 28, 2021, 6:01 AM Kevin Wolf wrote: > Am 27.10.2021 um 19:49 hat John Snow geschrieben: > > This reply is long, sorry. > > And after writing half of a very long reply myself, I noticed that I was > just very confused, so sorry for making you write a long text fo

Re: [PATCH v5 0/8] Switch iotests to using Async QMP

2021-10-28 Thread John Snow
On Thu, Oct 28, 2021 at 6:37 AM Kevin Wolf wrote: > Am 26.10.2021 um 19:56 hat John Snow geschrieben: > > GitLab: > https://gitlab.com/jsnow/qemu/-/commits/python-aqmp-iotest-wrapper > > CI: https://gitlab.com/jsnow/qemu/-/pipelines/395925703 > > > > Hiya, >

Re: [PATCH v5 6/8] iotests/300: avoid abnormal shutdown race condition

2021-10-27 Thread John Snow
On Wed, Oct 27, 2021 at 8:56 AM Kevin Wolf wrote: > Am 26.10.2021 um 19:56 hat John Snow geschrieben: > > Wait for the destination VM to close itself instead of racing to shut it > > down first, which produces different error log messages from AQMP > > depending on pr

Re: [PATCH v5 2/8] python/machine: Handle QMP errors on close more meticulously

2021-10-27 Thread John Snow
This reply is long, sorry. On Wed, Oct 27, 2021 at 7:19 AM Kevin Wolf wrote: > Am 26.10.2021 um 19:56 hat John Snow geschrieben: > > To use the AQMP backend, Machine just needs to be a little more diligent > > about what happens when closing a QMP connection. The operation i

Re: [PATCH v2 12/15] iotests/linters: Add entry point for linting via Python CI

2021-10-26 Thread John Snow
On Tue, Oct 26, 2021 at 2:36 PM John Snow wrote: > > > On Tue, Oct 26, 2021 at 6:57 AM Hanna Reitz wrote: > >> On 19.10.21 16:49, John Snow wrote: >> > We need at least a tiny little shim here to join test file discovery >> > with test invocation. Th

Re: [PATCH v2 0/2] pylint: fix new errors and warnings in qemu-iotests

2021-10-26 Thread John Snow
On Mon, Oct 11, 2021 at 5:58 AM Emanuele Giuseppe Esposito < eespo...@redhat.com> wrote: > > > On 11/10/2021 11:29, Hanna Reitz wrote: > > On 08.10.21 08:28, Emanuele Giuseppe Esposito wrote: > >> There are some warnings and errors that we either miss or > >> are new in pylint. Anyways, test 297

Re: [PATCH v2 12/15] iotests/linters: Add entry point for linting via Python CI

2021-10-26 Thread John Snow
On Tue, Oct 26, 2021 at 6:57 AM Hanna Reitz wrote: > On 19.10.21 16:49, John Snow wrote: > > We need at least a tiny little shim here to join test file discovery > > with test invocation. This logic could conceivably be hosted somewhere > > in python/, but I felt it was st

Re: [PATCH v2 11/15] iotests: split linters.py out from 297

2021-10-26 Thread John Snow
On Tue, Oct 26, 2021 at 6:51 AM Hanna Reitz wrote: > On 19.10.21 16:49, John Snow wrote: > > Now, 297 is just the iotests-specific incantations and linters.py is as > > minimal as I can think to make it. The only remaining element in here > > that ought to be confi

[PATCH v5 6/8] iotests/300: avoid abnormal shutdown race condition

2021-10-26 Thread John Snow
, which will cause an ECONNRESET error to be logged. Waiting for the VM to exit itself avoids the race on shutdown behavior.) Reported-by: Hanna Reitz Signed-off-by: John Snow --- tests/qemu-iotests/300 | 13 + 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/tests/qemu

Re: [PATCH v2 08/15] iotests/297: Change run_linter() to raise an exception on failure

2021-10-26 Thread John Snow
On Tue, Oct 26, 2021 at 6:10 AM Hanna Reitz wrote: > On 19.10.21 16:49, John Snow wrote: > > Instead of using a process return code as the python function return > > value (or just not returning anything at all), allow run_linter() to > > raise an exception instead. >

[PATCH v5 7/8] python/aqmp: Create sync QMP wrapper for iotests

2021-10-26 Thread John Snow
This is a wrapper around the async QMPClient that mimics the old, synchronous QEMUMonitorProtocol class. It is designed to be interchangeable with the old implementation. It does not, however, attempt to mimic Exception compatibility. Signed-off-by: John Snow Acked-by: Hanna Reitz --- python

[PATCH v5 5/8] iotests: Conditionally silence certain AQMP errors

2021-10-26 Thread John Snow
can't amend the iotest output to just always expect the error message, either. Just suppress it for now. More rigorous log filtering can be investigated later if/when it is deemed safe to permanently replace the legacy QMP library.) Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests

[PATCH v5 2/8] python/machine: Handle QMP errors on close more meticulously

2021-10-26 Thread John Snow
' command. That error will ultimately be bubbled up when attempting to close the QMP connection. The manager class here then is free to discard it -- if it was expected. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- python/qemu/machine/machine.py | 48 +- 1

Re: [PATCH v2 10/15] iotests/297: split test into sub-cases

2021-10-26 Thread John Snow
On Tue, Oct 26, 2021 at 6:29 AM Hanna Reitz wrote: > On 19.10.21 16:49, John Snow wrote: > > Take iotest 297's main() test function and split it into two sub-cases > > that can be skipped individually. We can also drop custom environment > > setup from the pylint te

[PATCH v5 4/8] iotests: Accommodate async QMP Exception classes

2021-10-26 Thread John Snow
. That'd be these: - QMPError - QMPConnectError - QMPCapabilitiesError - QMPTimeoutError - QMPProtocolError - QMPResponseError - QMPBadPortError Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- scripts/simplebench/bench_block_job.py| 3 ++- tests/qemu-iotests/tests/mirror-top-perms | 5

[PATCH v5 3/8] python/aqmp: Remove scary message

2021-10-26 Thread John Snow
The scary message interferes with the iotests output. Coincidentally, if iotests works by removing this, then it's good evidence that we don't really need to scare people away from using it. Signed-off-by: John Snow Acked-by: Hanna Reitz --- python/qemu/aqmp/__init__.py | 12 1

[PATCH v5 1/8] python/machine: remove has_quit argument

2021-10-26 Thread John Snow
ut down. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- python/qemu/machine/machine.py | 34 +++--- tests/qemu-iotests/040 | 7 +-- tests/qemu-iotests/218 | 2 +- tests/qemu-iotests/255 | 2 +- 4 files changed, 22 insertions(+), 2

[PATCH v5 0/8] Switch iotests to using Async QMP

2021-10-26 Thread John Snow
: Apocrypha. John Snow (8): python/machine: remove has_quit argument python/machine: Handle QMP errors on close more meticulously python/aqmp: Remove scary message iotests: Accommodate async QMP Exception classes iotests: Conditionally silence certain AQMP errors iotests/300: avoid abnormal

[PATCH v5 8/8] python, iotests: replace qmp with aqmp

2021-10-26 Thread John Snow
work concurrently. Signed-off-by: John Snow --- python/qemu/machine/machine.py | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index a0cf69786b4..a487c397459 100644 --- a/python/qemu/machine/machine.py +++ b

Re: [PATCH v4 6/8] iotests/300: avoid abnormal shutdown race condition

2021-10-26 Thread John Snow
On Mon, Oct 25, 2021 at 9:20 AM Hanna Reitz wrote: > On 13.10.21 23:57, John Snow wrote: > > Wait for the destination VM to close itself instead of racing to shut it > > down first, which produces different error log messages from AQMP > > depending on precisely when we

Re: [PATCH 2/9] qapi: Mark unstable QMP parts with feature 'unstable'

2021-10-26 Thread John Snow
On Tue, Oct 26, 2021 at 3:56 AM Markus Armbruster wrote: > John Snow writes: > > > On Mon, Oct 25, 2021 at 1:25 AM Markus Armbruster > wrote: > > > >> Add special feature 'unstable' everywhere the name starts with 'x-', > >> except for

Re: [PATCH 7/9] qapi: Generalize enum member policy checking

2021-10-26 Thread John Snow
On Tue, Oct 26, 2021 at 5:43 AM Markus Armbruster wrote: > John Snow writes: > > > On Mon, Oct 25, 2021 at 1:26 AM Markus Armbruster > wrote: > > > >> The code to check enumeration value policy can see special feature > >> flag 'deprecated' in QEnumLook

Re: [PATCH 7/9] qapi: Generalize enum member policy checking

2021-10-25 Thread John Snow
ret += mcgen(''' > }, > -.flags = (const unsigned char[%(max_index)s]) { > +.special_features = (const unsigned char[%(max_index)s]) { > ''', > max_index=max_index) > -ret += flags > +ret += feats > > ret += mcgen(''' > }, > -- > 2.31.1 > > Python bits: Acked-by: John Snow

Re: [PATCH 5/9] qapi: Generalize struct member policy checking

2021-10-25 Thread John Snow
On Mon, Oct 25, 2021 at 1:25 AM Markus Armbruster wrote: > The generated visitor functions call visit_deprecated_accept() and > visit_deprecated() when visiting a struct member with special feature > flag 'deprecated'. This makes the feature flag visible to the actual > visitors. I want to

Re: [PATCH 9/9] qapi: Extend -compat to set policy for unstable interfaces

2021-10-25 Thread John Snow
api/schema.py > +++ b/scripts/qapi/schema.py > @@ -254,9 +254,11 @@ def doc_type(self): > > def check(self, schema): > QAPISchemaEntity.check(self, schema) > -if 'deprecated' in [f.name for f in self.features]: > -raise QAPISemError( > -

Re: [PATCH 8/9] qapi: Factor out compat_policy_input_ok()

2021-10-25 Thread John Snow
On Mon, Oct 25, 2021 at 1:25 AM Markus Armbruster wrote: > The code to check policy for handling deprecated input is triplicated. > Factor it out into compat_policy_input_ok() before I mess with it in > the next commit. > > Signed-off-by: Markus Armbruster > (Skipping C-only patches for quick

Re: [PATCH 6/9] qapi: Generalize command policy checking

2021-10-25 Thread John Snow
options += ['QCO_DEPRECATED'] > - > if not success_response: > options += ['QCO_NO_SUCCESS_RESP'] > if allow_oob: > @@ -231,10 +229,11 @@ def gen_register_command(name: str, > > ret = mcgen(''' > qmp_register_command(cmds, "%(name)s", > - qmp_marshal_%(c_name)s, %(opts)s); > + qmp_marshal_%(c_name)s, %(opts)s, %(feats)s); > ''', > name=name, c_name=c_name(name), > -opts=' | '.join(options) or 0) > +opts=' | '.join(options) or 0, > +feats=gen_special_features(features)) > Ah, you use the '0' return here. Alright then. > return ret > > > -- > 2.31.1 > > Python bits: Acked-by: John Snow C bits: "I believe in my heart that they probably work." (for this and previous patch.)

Re: [PATCH 4/9] qapi: Tools for sets of special feature flags in generated code

2021-10-25 Thread John Snow
ial(self): > +return self.name in ('deprecated') > + > alrighty. (Briefly wondered: is it worth naming special features as a property of the class, but with only two names, it's probably fine enough to leave it embedded in the method logic. Only a style thing and doesn't have any actual impact that I can imagine, so ... nevermind.) > > class QAPISchemaObjectTypeMember(QAPISchemaMember): > def __init__(self, name, info, typ, optional, ifcond=None, > features=None): > -- > 2.31.1 > > Well, either way: Reviewed-by: John Snow

Re: [PATCH 2/9] qapi: Mark unstable QMP parts with feature 'unstable'

2021-10-25 Thread John Snow
for ramblock-id. Disable this > for 4.0 > #machine types or older to allow > # migration with newer QEMU > versions. > -#This option is considered stable > -#despite the x- prefix. (default: > -#false generally, but true for > machine > -#types <= 4.0) > +#(default: false generally, > +#but true for machine types <= > 4.0) > # > # Note: prealloc=true and reserve=false cannot be set at the same time. > With > # reserve=true, the behavior depends on the operating system: for > example, > @@ -785,6 +783,9 @@ > ## > # @ObjectType: > # > +# Features: > +# @unstable: Member @x-remote-object is experimental. > +# > # Since: 6.0 > ## > { 'enum': 'ObjectType', > @@ -836,7 +837,7 @@ > 'tls-creds-psk', > 'tls-creds-x509', > 'tls-cipher-suites', > -'x-remote-object' > +{ 'name': 'x-remote-object', 'features': [ 'unstable' ] } >] } > > ## > -- > 2.31.1 > > Seems OK, but I didn't audit for false positives/negatives. Trusting your judgment here. (It looks like Phil started to audit this in his reply to your previous commit, so I'll trust that.) Acked-by: John Snow

Re: [PATCH 3/9] qapi: Eliminate QCO_NO_OPTIONS for a slight simplification

2021-10-25 Thread John Snow
opts=" | ".join(options)) > +opts=' | '.join(options) or 0) > return ret > > > I'm not a big fan of naked constants on the C side, but the generator simplification is nice. I suppose it's worth the trade-off if you like it better this way. "eh". Reviewed-by: John Snow

Re: [PATCH 1/9] qapi: New special feature flag "unstable"

2021-10-25 Thread John Snow
and-features3 None -> None > gen=True success_response=True boxed=False oob=False preconfig=False > +feature unstable > feature feature1 > feature feature2 > command test-command-cond-features1 None -> None > @@ -394,6 +396,9 @@ event TEST_EVENT_FEATURES0 FeatureStruct1 > event TEST_EVENT_FEATURES1 None > boxed=False > feature deprecated > +event TEST_EVENT_FEATURES2 None > +boxed=False > +feature unstable > module include/sub-module.json > include sub-sub-module.json > object SecondArrayRef > -- > 2.31.1 > > Feels odd to combine the doc update *and* test prep, but eh, whatever. Reviewed-by: John Snow

[PATCH v2 15/15] iotests: [RFC] drop iotest 297

2021-10-19 Thread John Snow
running logic anymore is the skip list and the python-test-detection code. Without those, we could easily codify the tests as simply: [pylint|mypy] *.py tests/*.py ... and drop this entire file. We're not quite there yet, though. Signed-off-by: John Snow --- tests/qemu-iotests/297| 82

[PATCH v2 13/15] iotests/linters: Add workaround for mypy bug #9852

2021-10-19 Thread John Snow
es that 0.920 may bring with it.) See also: https://github.com/python/mypy/issues/11010 https://github.com/python/mypy/issues/9852 Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/linters.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/

[PATCH v2 11/15] iotests: split linters.py out from 297

2021-10-19 Thread John Snow
configurations both would be ... a hassle. Signed-off-by: John Snow --- tests/qemu-iotests/297| 72 + tests/qemu-iotests/linters.py | 76 +++ 2 files changed, 87 insertions(+), 61 deletions(-) create mode 100644 tests/qemu-iotests

[PATCH v2 10/15] iotests/297: split test into sub-cases

2021-10-19 Thread John Snow
Take iotest 297's main() test function and split it into two sub-cases that can be skipped individually. We can also drop custom environment setup from the pylint test as it isn't needed. Signed-off-by: John Snow --- tests/qemu-iotests/297 | 63 ++ 1 file

[PATCH v2 12/15] iotests/linters: Add entry point for linting via Python CI

2021-10-19 Thread John Snow
shebang.) Signed-off-by: John Snow --- tests/qemu-iotests/linters.py | 27 +++ 1 file changed, 27 insertions(+) diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters.py index c515c7afe36..46c28fdcda0 100644 --- a/tests/qemu-iotests/linters.py +++ b

[PATCH v2 09/15] iotests/297: update tool availability checks

2021-10-19 Thread John Snow
As mentioned in 'iotests/297: Don't rely on distro-specific linter binaries', these checks are overly strict. Update them to be in-line with how we actually invoke the linters themselves. Signed-off-by: John Snow --- tests/qemu-iotests/297 | 9 + 1 file changed, 5 insertions(+), 4

[PATCH v2 08/15] iotests/297: Change run_linter() to raise an exception on failure

2021-10-19 Thread John Snow
/format that information. (Also, "suppress_output" is now a lot more accurate of a parameter name.) Signed-off-by: John Snow --- tests/qemu-iotests/297 | 24 ++-- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests

[PATCH v2 04/15] iotests/297: Create main() function

2021-10-19 Thread John Snow
Instead of running "run_linters" directly, create a main() function that will be responsible for environment setup, leaving run_linters() responsible only for execution of the linters. (That environment setup will be moved over in forthcoming commits.) Signed-off-by: John Snow

[PATCH v2 02/15] iotests/297: Split mypy configuration out into mypy.ini

2021-10-19 Thread John Snow
More separation of code and configuration. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/297 | 14 +- tests/qemu-iotests/mypy.ini | 12 2 files changed, 13 insertions(+), 13 deletions(-) create mode 100644 tests/qemu-iotests/mypy.ini

[PATCH v2 01/15] iotests/297: Move pylint config into pylintrc

2021-10-19 Thread John Snow
Move --score=n and --notes=XXX,FIXME into pylintrc. This pulls configuration out of code, which I think is probably a good thing in general. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/297 | 4 +--- tests/qemu-iotests/pylintrc | 16 2 files

[PATCH v2 14/15] python: Add iotest linters to test suite

2021-10-19 Thread John Snow
against a wide variety of python versions, which helps to find accidental cross-version python compatibility issues. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- python/tests/iotests-mypy.sh | 4 python/tests/iotests-pylint.sh | 4 2 files changed, 8 insertions(+) create mode

[PATCH v2 06/15] iotests/297: Split run_linters apart into run_pylint and run_mypy

2021-10-19 Thread John Snow
Move environment setup into main(), and split the actual linter execution into run_pylint and run_mypy, respectively. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/297 | 38 -- 1 file changed, 24 insertions(+), 14 deletions

[PATCH v2 07/15] iotests/297: refactor run_[mypy|pylint] as generic execution shim

2021-10-19 Thread John Snow
There's virtually nothing special here anymore; we can combine these into a single, rather generic function. Signed-off-by: John Snow --- tests/qemu-iotests/297 | 42 ++ 1 file changed, 22 insertions(+), 20 deletions(-) diff --git a/tests/qemu-iotests

[PATCH v2 05/15] iotests/297: Don't rely on distro-specific linter binaries

2021-10-19 Thread John Snow
test begins -- this check is now "overly strict", but shouldn't cause anything that was already running correctly to start failing. This is addressed by a commit later in this series; 'iotests/297: update tool availability checks'. Signed-off-by: John Snow Reviewed-by: Vladimir Sement

[PATCH v2 03/15] iotests/297: Add get_files() function

2021-10-19 Thread John Snow
Split out file discovery into its own method to begin separating out configuration/setup and test execution. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/297 | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests

[PATCH v2 00/15] python/iotests: Run iotest linters during Python CI

2021-10-19 Thread John Snow
ng. - Added patch 8, which scans subdirs for tests to lint. - Added patch 17, which improves the speed of mypy analysis. - Patch 14 is different because of the new patch 8. John Snow (15): iotests/297: Move pylint config into pylintrc iotests/297: Split mypy configuration out into mypy

Re: iotest 030 SIGSEGV

2021-10-14 Thread John Snow
On Thu, Oct 14, 2021 at 9:20 AM Hanna Reitz wrote: > On 13.10.21 23:50, John Snow wrote: > > In trying to replace the QMP library backend, I have now twice > > stumbled upon a SIGSEGV in iotest 030 in the last three weeks or so. > > > > I didn't have debug symbols

[PATCH v4 2/8] python/machine: Handle QMP errors on close more meticulously

2021-10-13 Thread John Snow
' command. That error will ultimately be bubbled up when attempting to close the QMP connection. The manager class here then is free to discard it -- if it was expected. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- python/qemu/machine/machine.py | 48 +- 1

[PATCH v4 7/8] python/aqmp: Create sync QMP wrapper for iotests

2021-10-13 Thread John Snow
This is a wrapper around the async QMPClient that mimics the old, synchronous QEMUMonitorProtocol class. It is designed to be interchangeable with the old implementation. It does not, however, attempt to mimic Exception compatibility. Signed-off-by: John Snow Acked-by: Hanna Reitz --- python

[PATCH v4 8/8] python, iotests: replace qmp with aqmp

2021-10-13 Thread John Snow
work concurrently. Signed-off-by: John Snow --- python/qemu/machine/machine.py | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index a0cf69786b4..a487c397459 100644 --- a/python/qemu/machine/machine.py +++ b

[PATCH v4 4/8] iotests: Accommodate async QMP Exception classes

2021-10-13 Thread John Snow
. That'd be these: - QMPError - QMPConnectError - QMPCapabilitiesError - QMPTimeoutError - QMPProtocolError - QMPResponseError - QMPBadPortError Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- scripts/simplebench/bench_block_job.py| 3 ++- tests/qemu-iotests/tests/mirror-top-perms | 5

[PATCH v4 3/8] python/aqmp: Remove scary message

2021-10-13 Thread John Snow
The scary message interferes with the iotests output. Coincidentally, if iotests works by removing this, then it's good evidence that we don't really need to scare people away from using it. Signed-off-by: John Snow Acked-by: Hanna Reitz --- python/qemu/aqmp/__init__.py | 12 1

[PATCH v4 5/8] iotests: Conditionally silence certain AQMP errors

2021-10-13 Thread John Snow
can't amend the iotest output to just always expect the error message, either. Just suppress it for now. More rigorous log filtering can be investigated later if/when it is deemed safe to permanently replace the legacy QMP library.) Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests

[PATCH v4 6/8] iotests/300: avoid abnormal shutdown race condition

2021-10-13 Thread John Snow
, which will cause an ECONNRESET error to be logged. Waiting for the VM to exit itself avoids the race on shutdown behavior.) Reported-by: Hanna Reitz Signed-off-by: John Snow --- tests/qemu-iotests/300 | 12 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/tests/qemu-iotests

[PATCH v4 0/8] Switch iotests to using Async QMP

2021-10-13 Thread John Snow
ut of 0.0 will cause a timeout exception to be raised even if there was an event ready to be read. V2: A distant dream, half-remembered. V1: Apocrypha. John Snow (8): python/machine: remove has_quit argument python/machine: Handle QMP errors on close more meticulously python/aqmp:

[PATCH v4 1/8] python/machine: remove has_quit argument

2021-10-13 Thread John Snow
ut down. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- python/qemu/machine/machine.py | 34 +++--- tests/qemu-iotests/040 | 7 +-- tests/qemu-iotests/218 | 2 +- tests/qemu-iotests/255 | 2 +- 4 files changed, 22 insertions(+), 2

iotest 030 SIGSEGV

2021-10-13 Thread John Snow
In trying to replace the QMP library backend, I have now twice stumbled upon a SIGSEGV in iotest 030 in the last three weeks or so. I didn't have debug symbols on at the time, so I've got only this stack trace: (gdb) thread apply all bt Thread 8 (Thread 0x7f0a6b8c4640 (LWP 1873554)): #0

Re: [PATCH v3 0/7] Switch iotests to using Async QMP

2021-10-13 Thread John Snow
On Wed, Oct 13, 2021 at 10:49 AM Hanna Reitz wrote: > On 13.10.21 16:00, John Snow wrote: > > > > > > On Wed, Oct 13, 2021 at 8:51 AM John Snow wrote: > > > > > > > > On Wed, Oct 13, 2021 at 4:45 AM Hanna Reitz > wrote: &g

Re: [PATCH 10/13] iotests/linters: Add entry point for linting via Python CI

2021-10-13 Thread John Snow
On Wed, Oct 13, 2021 at 8:11 AM Hanna Reitz wrote: > On 04.10.21 23:05, John Snow wrote: > > We need at least a tiny little shim here to join test file discovery > > with test invocation. This logic could conceivably be hosted somewhere > > in python/, but I felt it was st

Re: [PATCH 09/13] iotests: split linters.py out from 297

2021-10-13 Thread John Snow
On Wed, Oct 13, 2021 at 7:50 AM Hanna Reitz wrote: > On 04.10.21 23:04, John Snow wrote: > > Now, 297 is just the iotests-specific incantations and linters.py is as > > minimal as I can think to make it. The only remaining element in here > > that ought to be confi

Re: [PATCH 05/13] iotests/297: Create main() function

2021-10-13 Thread John Snow
On Wed, Oct 13, 2021 at 7:03 AM Hanna Reitz wrote: > On 04.10.21 23:04, John Snow wrote: > > Instead of running "run_linters" directly, create a main() function that > > will be responsible for environment setup, leaving run_linters() > > responsible on

Re: [PATCH 02/13] iotests/297: Split mypy configuration out into mypy.ini

2021-10-13 Thread John Snow
On Wed, Oct 13, 2021 at 6:53 AM Hanna Reitz wrote: > On 04.10.21 23:04, John Snow wrote: > > More separation of code and configuration. > > > > Signed-off-by: John Snow > > --- > > tests/qemu-iotests/297 | 14 +- > > tests/qemu-iot

Re: [PATCH v3 0/7] Switch iotests to using Async QMP

2021-10-13 Thread John Snow
On Wed, Oct 13, 2021 at 8:51 AM John Snow wrote: > > > On Wed, Oct 13, 2021 at 4:45 AM Hanna Reitz wrote: > >> On 13.10.21 00:34, John Snow wrote: >> > Based-on: <20211012214152.802483-1-js...@redhat.com> >> >[PULL 00/10] Python patches >

Re: [PATCH v3 0/7] Switch iotests to using Async QMP

2021-10-13 Thread John Snow
On Wed, Oct 13, 2021 at 4:45 AM Hanna Reitz wrote: > On 13.10.21 00:34, John Snow wrote: > > Based-on: <20211012214152.802483-1-js...@redhat.com> > >[PULL 00/10] Python patches > > GitLab: > https://gitlab.com/jsnow/qemu/-/commits/python-aqmp-iotest-wrap

[PATCH v3 2/7] python/machine: Handle QMP errors on close more meticulously

2021-10-12 Thread John Snow
' command. That error will ultimately be bubbled up when attempting to close the QMP connection. The manager class here then is free to discard it -- if it was expected. Signed-off-by: John Snow --- python/qemu/machine/machine.py | 48 +- 1 file changed, 42

<    4   5   6   7   8   9   10   11   12   13   >