[PATCH RFC 23/32] python//machine.py: reorder __init__

2020-05-13 Thread John Snow
Put the init arg handling all at the top, and mostly in order (deviating when one is dependent on another), and put what is effectively runtime state declaration at the bottom. Signed-off-by: John Snow --- python/qemu/lib/machine.py | 29 + 1 file changed, 17

[PATCH RFC 18/32] python//qmp.py: add casts to JSON deserialization

2020-05-13 Thread John Snow
mypy and python type hints are not powerful enough to properly describe JSON messages in Python 3.6. The best we can do, generally, is describe them as Dict[str, Any]. Add casts to coerce this type for static analysis; but do NOT enforce this type at runtime in any way. Note: Python 3.8 adds a

[PATCH RFC 16/32] python//qmp.py: re-absorb MonitorResponseError

2020-05-13 Thread John Snow
When I initially split this out, I considered this more of a machine error than a QMP protocol error, but I think that's misguided. Move this back to qmp.py and name it QMPResponseError. Convert qmp.command() to use this exception type. Signed-off-by: John Snow --- python/qemu/lib/machine.py

[PATCH RFC 15/32] python//qmp.py: Define common types

2020-05-13 Thread John Snow
Define some common types that we'll need to annotate a lot of other functions going forward. Signed-off-by: John Snow --- python/qemu/lib/qmp.py | 18 ++ 1 file changed, 18 insertions(+) diff --git a/python/qemu/lib/qmp.py b/python/qemu/lib/qmp.py index a634c4e26c..911da59888

[PATCH RFC 20/32] python//qmp.py: assert sockfile is not None

2020-05-13 Thread John Snow
In truth, if you don't do this, you'll just get a TypeError exception. Now, you'll get an AssertionError. Is this tangibly better? No. Does mypy complain less? Yes. Signed-off-by: John Snow --- python/qemu/lib/qmp.py | 1 + 1 file changed, 1 insertion(+) diff --git a/python/qemu/lib/qmp.py

[PATCH RFC 22/32] python//machine.py: Fix monitor address typing

2020-05-13 Thread John Snow
Prior to this, it's difficult for mypy to intuit what the concrete type of the monitor address is; it has difficulty inferring the type across two variables. Create _monitor_address as a property that always returns a valid address to simply static type analysis. To preserve our ability to clean

[PATCH RFC 06/32] python/qemu: formalize as package

2020-05-13 Thread John Snow
NB: I am choosing Python 3.6 here. Although our minimum requirement is 3.5, this code is used only by iotests (so far) under which we have been using a minimum version of 3.6. 3.6 is being preferred here for variable type hint capability, which enables us to use mypy for this package.

[PATCH RFC 05/32] python/qemu/lib: delint; add flake8 config

2020-05-13 Thread John Snow
Mostly, ignore the "no bare except" rule, because flake8 is not contextual and cannot determine if we re-raise. Pylint can, though, so always prefer pylint for that. Signed-off-by: John Snow --- python/qemu/lib/.flake8| 2 ++ python/qemu/lib/accel.py | 9 ++---

[PATCH RFC 08/32] python/qemu: Add Pipfile

2020-05-13 Thread John Snow
pipenv is a tool used for managing virtual environments with precisely specified dependencies. It is separate from the dependencies listed in setup.py, which are not pinned to precise versions. Note that pipenv is not required to install or use this module; this is just a convenience for in-tree

[PATCH RFC 04/32] python/qemu/lib: delint, add pylintrc

2020-05-13 Thread John Snow
Bring our these files up to speed with pylint 2.5.0. Add a pylintrc file to formalize which pylint subset we are targeting. The similarity ignore is there to suppress similarity reports across imports, which for typing constants, are going to trigger this report erroneously. Signed-off-by: John

[PATCH RFC 07/32] python/qemu: add README.rst

2020-05-13 Thread John Snow
Add a short readme that explains the package hierarchy, which will be visible while browsing the source on e.g. gitlab/github. Signed-off-by: John Snow --- python/qemu/README.rst | 8 1 file changed, 8 insertions(+) create mode 100644 python/qemu/README.rst diff --git

[PATCH RFC 10/32] python/qemu: Add flake8 to Pipfile

2020-05-13 Thread John Snow
Signed-off-by: John Snow --- python/Pipfile | 1 + python/Pipfile.lock | 31 ++- 2 files changed, 31 insertions(+), 1 deletion(-) diff --git a/python/Pipfile b/python/Pipfile index ddb2b5a518..e396e56e06 100644 --- a/python/Pipfile +++ b/python/Pipfile @@ -5,6

[PATCH RFC 14/32] python//qmp.py: use True/False for non/blocking modes

2020-05-13 Thread John Snow
The type system doesn't want integers. Signed-off-by: John Snow --- python/qemu/lib/qmp.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/python/qemu/lib/qmp.py b/python/qemu/lib/qmp.py index b91c9d5c1c..a634c4e26c 100644 --- a/python/qemu/lib/qmp.py +++

[PATCH RFC 02/32] scripts/qmp: Fix shebang and imports

2020-05-13 Thread John Snow
There's more wrong with these scripts; They are in various stages of disrepair. That's beyond the scope of this current patchset. This just mechanically corrects the imports and the shebangs, as part of ensuring that the python/qemu/lib refactoring didn't break anything needlessly.

[PATCH RFC 11/32] python/qemu/lib: remove Python2 style super() calls

2020-05-13 Thread John Snow
Use the Python3 style instead. Signed-off-by: John Snow --- python/qemu/lib/machine.py | 2 +- python/qemu/lib/qtest.py | 15 +++ 2 files changed, 8 insertions(+), 9 deletions(-) diff --git a/python/qemu/lib/machine.py b/python/qemu/lib/machine.py index 4b260fa2cb..b2f0412197

[PATCH RFC 03/32] python//machine.py: remove bare except

2020-05-13 Thread John Snow
Catch only the timeout error; if there are other problems, allow the stack trace to be visible. Signed-off-by: John Snow --- python/qemu/lib/machine.py | 33 + 1 file changed, 21 insertions(+), 12 deletions(-) diff --git a/python/qemu/lib/machine.py

[PATCH RFC 01/32] python/qemu: create qemu.lib module

2020-05-13 Thread John Snow
move python/qemu/*.py to python/qemu/lib/*.py. To create a namespace package, the 'qemu' directory itself shouldn't have module files in it. Thus, these files will go under a 'lib' package directory instead. Bolster the lib/__init__.py file a little bit, Make the top-level classes and functions

[PATCH RFC 00/32] python/qemu: refactor as installable package

2020-05-13 Thread John Snow
Hey, I got lost on my way to the store and I accidentally got 32 patches that convert our python library into something that passes pylint, flake8, and mypy --strict. ...So, a few things: 1. This is just an RFC. The actual design of these libraries really needs adjusted to be more pythonic. In

Re: [PATCH v5 08/15] acpi: move aml builder code for floppy device

2020-05-13 Thread Thomas Huth
On 13/05/2020 22.43, John Snow wrote: > > > On 5/7/20 10:05 AM, Philippe Mathieu-Daudé wrote: >> +Hervé >> >> On 5/7/20 3:16 PM, Gerd Hoffmann wrote: >>> Signed-off-by: Gerd Hoffmann >>> Reviewed-by: Igor Mammedov >>> --- >>>   hw/block/fdc.c   | 83

Re: [PATCH v4 4/9] blockdev: Promote several bitmap functions to non-static

2020-05-13 Thread Vladimir Sementsov-Ogievskiy
13.05.2020 04:16, Eric Blake wrote: The next patch will split blockdev.c, which will require accessing some previously-static functions from more than one .c file. But part of promoting a function to public is picking a naming scheme that does not reek of exposing too many internals (two of the

Re: [PATCH] qemu-nbd: Close inherited stderr

2020-05-13 Thread Raphael Pour
On 5/13/20 3:02 PM, Eric Blake wrote: > Yes, now that we know about it, the bug will be fixed in 5.1; we can > also cc: qemu-stable to get it backported to the next 5.0.x release > (downstream developers are also more likely to backport it to their > ports as well if it lands on qemu-stable). 

Re: [PATCH v3 06/10] iotests: add testfinder.py

2020-05-13 Thread Vladimir Sementsov-Ogievskiy
14.05.2020 08:06, John Snow wrote: On 5/14/20 12:54 AM, Vladimir Sementsov-Ogievskiy wrote: 14.05.2020 00:58, John Snow wrote: On 5/7/20 1:43 PM, Vladimir Sementsov-Ogievskiy wrote: 21.04.2020 19:56, Kevin Wolf wrote: Am 21.04.2020 um 09:35 hat Vladimir Sementsov-Ogievskiy geschrieben:

Re: [PATCH v4 3/9] block: Make it easier to learn which BDS support bitmaps

2020-05-13 Thread Vladimir Sementsov-Ogievskiy
13.05.2020 04:16, Eric Blake wrote: Upcoming patches will enhance bitmap support in qemu-img, but in doing so, it turns out to be nice to suppress output when persistent bitmaps make no sense (such as on a qcow2 v2 image). Add a hook to make this easier to query. This patch adds a new callback

Re: [PATCH] bitmaps: Add myself as maintainer

2020-05-13 Thread John Snow
On 5/14/20 12:49 AM, Vladimir Sementsov-Ogievskiy wrote: > 13.05.2020 23:24, John Snow wrote: >> >> >> On 5/13/20 10:14 AM, Eric Blake wrote: >>> Dirty bitmaps are important to incremental backups, including exposure >>> over NBD where I'm already maintainer.  Also, I'm aware that lately I >>>

Re: [PATCH v3 06/10] iotests: add testfinder.py

2020-05-13 Thread John Snow
On 5/14/20 12:54 AM, Vladimir Sementsov-Ogievskiy wrote: > 14.05.2020 00:58, John Snow wrote: >> >> >> On 5/7/20 1:43 PM, Vladimir Sementsov-Ogievskiy wrote: >>> 21.04.2020 19:56, Kevin Wolf wrote: Am 21.04.2020 um 09:35 hat Vladimir Sementsov-Ogievskiy geschrieben: > Add python script

Re: [PATCH v4 2/9] qemu-img: Fix stale comments on doc location

2020-05-13 Thread Vladimir Sementsov-Ogievskiy
13.05.2020 04:16, Eric Blake wrote: Missed in commit e13c59fa. Signed-off-by: Eric Blake Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy -- Best regards, Vladimir

Re: [PATCH v4 1/9] docs: Sort sections on qemu-img subcommand parameters

2020-05-13 Thread Vladimir Sementsov-Ogievskiy
13.05.2020 04:16, Eric Blake wrote: We already list the subcommand summaries alphabetically, we should do the same for the documentation related to subcommand-specific parameters. Signed-off-by: Eric Blake Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy -- Best regards,

[PATCH v6 17/20] hw/block/nvme: factor out cmb setup

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch --- hw/block/nvme.c | 43 --- 1 file changed, 24 insertions(+), 19 deletions(-) diff --git a/hw/block/nvme.c

Re: [PATCH] bitmaps: Add myself as maintainer

2020-05-13 Thread Vladimir Sementsov-Ogievskiy
13.05.2020 23:24, John Snow wrote: On 5/13/20 10:14 AM, Eric Blake wrote: Dirty bitmaps are important to incremental backups, including exposure over NBD where I'm already maintainer. Also, I'm aware that lately I have been doing as much code/review on bitmaps as John Snow, who is hoping to

Re: [PATCH v3 06/10] iotests: add testfinder.py

2020-05-13 Thread Vladimir Sementsov-Ogievskiy
14.05.2020 00:58, John Snow wrote: On 5/7/20 1:43 PM, Vladimir Sementsov-Ogievskiy wrote: 21.04.2020 19:56, Kevin Wolf wrote: Am 21.04.2020 um 09:35 hat Vladimir Sementsov-Ogievskiy geschrieben: Add python script with new logic of searching for tests: Current ./check behavior:   - tests

[PATCH v6 12/20] hw/block/nvme: factor out device state setup

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch --- hw/block/nvme.c | 22 +- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index

[PATCH v6 18/20] hw/block/nvme: factor out pmr setup

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Maxim Levitsky --- hw/block/nvme.c | 95 ++--- 1 file changed, 51 insertions(+), 44 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index d71a5f142d51..7254b66ae199 100644 ---

[PATCH v6 13/20] hw/block/nvme: factor out block backend setup

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch --- hw/block/nvme.c | 13 ++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index

[PATCH v6 20/20] hw/block/nvme: factor out controller identify setup

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch --- hw/block/nvme.c | 49 ++--- 1 file changed, 26 insertions(+), 23 deletions(-) diff --git a/hw/block/nvme.c

[PATCH v6 19/20] hw/block/nvme: do cmb/pmr init as part of pci init

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Maxim Levitsky --- hw/block/nvme.c | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index 7254b66ae199..2addcc86034a 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c

[PATCH v6 11/20] hw/block/nvme: factor out property/constraint checks

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch --- hw/block/nvme.c | 48 ++-- 1 file changed, 30 insertions(+), 18 deletions(-) diff --git a/hw/block/nvme.c

[PATCH v6 15/20] hw/block/nvme: factor out namespace setup

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch --- hw/block/nvme.c | 46 ++ 1 file changed, 26 insertions(+), 20 deletions(-) diff --git a/hw/block/nvme.c

[PATCH v6 07/20] hw/block/nvme: fix pin-based interrupt behavior

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen First, since the device only supports MSI-X or pin-based interrupt, if MSI-X is not enabled, it should not accept interrupt vectors different from 0 when creating completion queues. Secondly, the irq_status NvmeCtrl member is meant to be compared to the INTMS register, so it

[PATCH v6 16/20] hw/block/nvme: factor out pci setup

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch --- hw/block/nvme.c | 30 ++ 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c

[PATCH v6 14/20] hw/block/nvme: add namespace helpers

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen Introduce some small helpers to make the next patches easier on the eye. Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch --- hw/block/nvme.c | 3 +-- hw/block/nvme.h | 17 + 2 files

[PATCH v6 09/20] hw/block/nvme: add max_ioqpairs device parameter

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen The num_queues device paramater has a slightly confusing meaning because it accounts for the admin queue pair which is not really optional. Secondly, it is really a maximum value of queues allowed. Add a new max_ioqpairs parameter that only accounts for I/O queue pairs, but

[PATCH v6 05/20] hw/block/nvme: use constants in identify

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Maxim Levitsky Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Keith Busch --- hw/block/nvme.c | 8 include/block/nvme.h | 8 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/hw/block/nvme.c

[PATCH v6 06/20] hw/block/nvme: refactor nvme_addr_read

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen Pull the controller memory buffer check to its own function. The check will be used on its own in later patches. Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch --- hw/block/nvme.c | 16

[PATCH v6 08/20] hw/block/nvme: allow use of any valid msix vector

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen If the device uses MSI-X, any of the 2048 MSI-X interrupt vectors are valid. If the device is not using MSI-X, vector will and can only be zero at this point. Cc: "Michael S. Tsirkin" Cc: Marcel Apfelbaum Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 2 +- 1 file

[PATCH v6 02/20] hw/block/nvme: rename trace events to pci_nvme

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen Change the prefix of all nvme device related trace events to 'pci_nvme' to not clash with trace events from the nvme block driver. Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch --- hw/block/nvme.c

[PATCH v6 03/20] hw/block/nvme: remove superfluous breaks

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen These break statements was left over when commit 3036a626e9ef ("nvme: add Get/Set Feature Timestamp support") was merged. Signed-off-by: Klaus Jensen Reviewed-by: Maxim Levitsky Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Keith Busch --- hw/block/nvme.c | 4 1

[PATCH v6 10/20] hw/block/nvme: remove redundant cmbloc/cmbsz members

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch --- hw/block/nvme.c | 7 ++- hw/block/nvme.h | 2 -- 2 files changed, 2 insertions(+), 7 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c

[PATCH v6 04/20] hw/block/nvme: move device parameters to separate struct

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen Move device configuration parameters to separate struct to make it explicit what is configurable and what is set internally. Signed-off-by: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky --- hw/block/nvme.c | 49

[PATCH v6 00/20] nvme: small fixes, refactoring and cleanups

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen Changes since v5 * Prefixed all patches with "hw/block/nvme" to avoid confusion with the nvme block driver. * Added patch two patches: hw/block/nvme: fix pin-based interrupt behavior hw/block/nvme: allow use of any valid msix vector These were

[PATCH v6 01/20] hw/block/nvme: fix pci doorbell size calculation

2020-05-13 Thread Klaus Jensen
From: Klaus Jensen The size of the BAR is 0x1000 (main registers) + 8 bytes for each queue. Currently, the size of the BAR is calculated like so: n->reg_size = pow2ceil(0x1004 + 2 * (n->num_queues + 1) * 4); Since the 'num_queues' parameter already accounts for the admin queue, this should

[PATCH] python: remove more instances of sys.version_info

2020-05-13 Thread John Snow
We guarantee 3.5+ everywhere; remove more dead checks. In general, try to avoid using version checks and instead prefer to attempt behavior when possible. Signed-off-by: John Snow --- scripts/analyze-migration.py | 5 - scripts/decodetree.py| 25

[PATCH RFC v2 2/5] blockdev: combine DriveBackupState and BlockdevBackupState

2020-05-13 Thread John Snow
They have the same fields -- rename it BlockJobState. Signed-off-by: John Snow --- blockdev.c | 30 -- 1 file changed, 12 insertions(+), 18 deletions(-) diff --git a/blockdev.c b/blockdev.c index b3c840ec03..d3e8a6ca22 100644 --- a/blockdev.c +++ b/blockdev.c @@

[PATCH RFC v2 4/5] iotests: move bitmap helpers into their own file

2020-05-13 Thread John Snow
Signed-off-by: John Snow --- tests/qemu-iotests/257| 110 +--- tests/qemu-iotests/bitmaps.py | 131 ++ 2 files changed, 132 insertions(+), 109 deletions(-) create mode 100644 tests/qemu-iotests/bitmaps.py diff --git

[PATCH RFC v2 1/5] block: add bitmap-populate job

2020-05-13 Thread John Snow
This job copies the allocation map into a bitmap. It's a job because there's no guarantee that allocation interrogation will be quick (or won't hang), so it cannot be retrofit into block-dirty-bitmap-merge. It was designed with different possible population patterns in mind, but only top layer

[PATCH RFC v2 3/5] qmp: expose block-dirty-bitmap-populate

2020-05-13 Thread John Snow
This is a new job-creating command. Signed-off-by: John Snow --- qapi/block-core.json | 18 +++ qapi/transaction.json | 2 ++ blockdev.c| 74 +++ 3 files changed, 94 insertions(+) diff --git a/qapi/block-core.json

[PATCH RFC v2 0/5] block: add block-dirty-bitmap-populate job

2020-05-13 Thread John Snow
Hi, This is a new (very small) block job that writes a pattern into a bitmap. The only pattern implemented is the top allocation information. This can be used to "recover" an incremental bitmap chain if an external snapshot was taken without creating a new bitmap first: any writes made to the

Re: [PATCH RFC WIP 0/6] iotests: delinting work-in-progress

2020-05-13 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200513214130.15375-1-js...@redhat.com/ Hi, This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN ===

[PATCH v4 3/3] iotests: modify test 040 to use JobRunner

2020-05-13 Thread John Snow
Instead of having somewhat reproduced it for itself. Signed-off-by: John Snow --- tests/qemu-iotests/040 | 51 +- 1 file changed, 25 insertions(+), 26 deletions(-) diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040 index 90b59081ff..e2ef3bb812

[PATCH v4 0/3] iotests: add JobRunner framework

2020-05-13 Thread John Snow
The basic idea is to make a generic job runtime manager and allow callers to subclass the manager. Then, instead of adding callback arguments to the function all the time, we have à la carte customization of the loop. To showcase this a little bit, I removed the pre_finalization argument and made

[PATCH v4 2/3] iotests: add JobRunner class

2020-05-13 Thread John Snow
The idea is that instead of increasing the arguments to job_run all the time, create a more general-purpose job runner that can be subclassed to do interesting things with. pylint note: the 'callbacks' option guards against unused warning arguments in functions designated as callbacks. It does

[PATCH v4 1/3] qmp.py: change event_wait to use a dict

2020-05-13 Thread John Snow
It's easier to work with than a list of tuples, because we can check the keys for membership. Signed-off-by: John Snow --- python/qemu/machine.py| 10 +- tests/qemu-iotests/040| 12 ++-- tests/qemu-iotests/260| 5 +++-- tests/qemu-iotests/iotests.py | 16

Re: [PATCH v3 0/4] Additional parameters for qemu_img map

2020-05-13 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200513133629.18508-1-eyal.moscov...@oracle.com/ Hi, This series failed the docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT

Re: [PATCH v3 06/10] iotests: add testfinder.py

2020-05-13 Thread John Snow
On 5/7/20 1:43 PM, Vladimir Sementsov-Ogievskiy wrote: > 21.04.2020 19:56, Kevin Wolf wrote: >> Am 21.04.2020 um 09:35 hat Vladimir Sementsov-Ogievskiy geschrieben: >>> Add python script with new logic of searching for tests: >>> >>> Current ./check behavior: >>>   - tests are named

[PATCH RFC WIP 3/6] nbd-fault-injector: delint

2020-05-13 Thread John Snow
Signed-off-by: John Snow --- tests/qemu-iotests/Makefile | 2 +- tests/qemu-iotests/nbd-fault-injector.py | 34 ++-- 2 files changed, 21 insertions(+), 15 deletions(-) diff --git a/tests/qemu-iotests/Makefile b/tests/qemu-iotests/Makefile index

[PATCH RFC WIP 0/6] iotests: delinting work-in-progress

2020-05-13 Thread John Snow
I ran out of time, but I was briefly entertaining the idea of hitting everything else in the iotests folder with the pylint and mypy beam. This is just a draft of what I had at the time, in case someone gets around to it before I do. I forced all of the python scripts in this directory to

[PATCH RFC WIP 2/6] Makefile: add delint WIP

2020-05-13 Thread John Snow
Signed-off-by: John Snow --- tests/qemu-iotests/Makefile | 6 ++ 1 file changed, 6 insertions(+) diff --git a/tests/qemu-iotests/Makefile b/tests/qemu-iotests/Makefile index 27380e60c1..7dbb7f0fff 100644 --- a/tests/qemu-iotests/Makefile +++ b/tests/qemu-iotests/Makefile @@ -1,3 +1,6 @@

[PATCH RFC WIP 1/6] iotests: type hint wip

2020-05-13 Thread John Snow
Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 28 ++-- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 6c0e781af7..27c477c8a7 100644 --- a/tests/qemu-iotests/iotests.py +++

[PATCH RFC WIP 5/6] qcow2.py: delint

2020-05-13 Thread John Snow
Signed-off-by: John Snow --- tests/qemu-iotests/Makefile | 5 +- tests/qemu-iotests/pylintrc | 1 + tests/qemu-iotests/qcow2.py | 156 +++- 3 files changed, 104 insertions(+), 58 deletions(-) diff --git a/tests/qemu-iotests/Makefile

[PATCH RFC WIP 4/6] qed.py: delint

2020-05-13 Thread John Snow
Signed-off-by: John Snow --- tests/qemu-iotests/Makefile | 2 +- tests/qemu-iotests/qed.py | 46 - 2 files changed, 31 insertions(+), 17 deletions(-) diff --git a/tests/qemu-iotests/Makefile b/tests/qemu-iotests/Makefile index 64db48342f..5a3a1e8092 100644

[PATCH RFC WIP 6/6] WIP: delint test files

2020-05-13 Thread John Snow
Signed-off-by: John Snow --- tests/qemu-iotests/Makefile | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/qemu-iotests/Makefile b/tests/qemu-iotests/Makefile index 77da9fd96d..fd85eb5bae 100644 --- a/tests/qemu-iotests/Makefile +++ b/tests/qemu-iotests/Makefile @@ -1,4 +1,6 @@

Re: [PATCH v4 34/34] block: Drop @child_class from bdrv_child_perm()

2020-05-13 Thread Eric Blake
On 5/13/20 6:05 AM, Max Reitz wrote: Implementations should decide the necessary permissions based on @role. Signed-off-by: Max Reitz --- +++ b/block.c @@ -1947,13 +1947,13 @@ bool bdrv_is_writable(BlockDriverState *bs) } static void bdrv_child_perm(BlockDriverState *bs,

Re: [PATCH v4 33/34] block: Pass BdrvChildRole in remaining cases

2020-05-13 Thread Eric Blake
On 5/13/20 6:05 AM, Max Reitz wrote: These calls have no real use for the child role yet, but it will not harm to give one. Notably, the bdrv_root_attach_child() call in blockjob.c is left unmodified because there is not much the generic BlockJob object wants from its children. Signed-off-by:

Re: [PATCH 4/4] scripts/qmp: Fix QEMU Python scripts path

2020-05-13 Thread John Snow
On 4/30/20 1:04 AM, Markus Armbruster wrote: > John Snow writes: > >> On 4/21/20 5:42 AM, Philippe Mathieu-Daudé wrote: >>> QEMU Python scripts have been moved in commit 8f8fd9edba4 ("Introduce >>> Python module structure"). Use the same sys.path modification used >>> in the referenced commit

Re: [PATCH v4 32/34] block: Drop child_file

2020-05-13 Thread Eric Blake
On 5/13/20 6:05 AM, Max Reitz wrote: Signed-off-by: Max Reitz --- include/block/block_int.h | 1 - block.c | 39 ++- tests/test-bdrv-drain.c | 8 +++- 3 files changed, 5 insertions(+), 43 deletions(-) Reviewed-by: Eric Blake

Re: [PATCH v4 27/34] block: Use child_of_bds in remaining places

2020-05-13 Thread Eric Blake
On 5/13/20 6:05 AM, Max Reitz wrote: Replace child_file by child_of_bds in all remaining places (excluding tests). Signed-off-by: Max Reitz --- block.c | 3 ++- block/backup-top.c | 4 ++-- block/blklogwrites.c | 4 ++-- block/raw-format.c | 15 +-- 4

Re: [PATCH v4 6/6] tests/migration/guestperf: Use Python 3 interpreter

2020-05-13 Thread John Snow
On 5/12/20 6:32 AM, Philippe Mathieu-Daudé wrote: > Signed-off-by: Philippe Mathieu-Daudé Reviewed-by: John Snow

Re: [PATCH v4 5/6] scripts/modules/module_block: Use Python 3 interpreter & add pseudo-main

2020-05-13 Thread John Snow
On 5/12/20 6:32 AM, Philippe Mathieu-Daudé wrote: > Signed-off-by: Philippe Mathieu-Daudé > --- > scripts/modules/module_block.py | 31 --- > 1 file changed, 16 insertions(+), 15 deletions(-) > > diff --git a/scripts/modules/module_block.py

Re: [PATCH v4 26/34] block: Make filter drivers use child_of_bds

2020-05-13 Thread Eric Blake
On 5/13/20 6:05 AM, Max Reitz wrote: Note that some filters have secondary children, namely blkverify (the image to be verified) and blklogwrites (the log). This patch does not touch those children. Note that for blkverify, the filtered child should not be format-probed. While there is nothing

Re: [PATCH v4 4/6] scripts/kvm/vmxcap: Use Python 3 interpreter and add pseudo-main()

2020-05-13 Thread John Snow
On 5/12/20 6:32 AM, Philippe Mathieu-Daudé wrote: > Signed-off-by: Philippe Mathieu-Daudé > --- > scripts/kvm/vmxcap | 7 --- > 1 file changed, 4 insertions(+), 3 deletions(-) > > diff --git a/scripts/kvm/vmxcap b/scripts/kvm/vmxcap > index 971ed0e721..6fe66d5f57 100755 > ---

Re: [PATCH v4 3/6] scripts/qmp: Use Python 3 interpreter

2020-05-13 Thread John Snow
On 5/12/20 6:32 AM, Philippe Mathieu-Daudé wrote: > From: Philippe Mathieu-Daudé > > Signed-off-by: Philippe Mathieu-Daudé Whoops, I address this in a patch series I'm working on, too. I'll keep my patch in there for now until this one makes it in, or vice-versa. Reviewed-by: John Snow >

Re: [PATCH v4 2/6] scripts/qemu-gdb: Use Python 3 interpreter

2020-05-13 Thread John Snow
On 5/12/20 6:32 AM, Philippe Mathieu-Daudé wrote: > From: Philippe Mathieu-Daudé > > Signed-off-by: Philippe Mathieu-Daudé Reviewed-by: John Snow

Re: [PATCH v4 1/6] scripts/qemugdb: Remove shebang header

2020-05-13 Thread John Snow
On 5/12/20 6:32 AM, Philippe Mathieu-Daudé wrote: > From: Philippe Mathieu-Daudé > > These scripts are loaded as plugin by GDB (and they don't > have any __main__ entry point). Remove the shebang header. > > Acked-by: Alex Bennée > Signed-off-by: Philippe Mathieu-Daudé

Re: [PATCH 0/2] iotests: Run pylint and mypy in a testcase

2020-05-13 Thread John Snow
On 5/11/20 12:35 PM, Kevin Wolf wrote: > Kevin Wolf (2): > iotests: Fix incomplete type declarations > iotests: Run pylint and mypy in a testcase > > tests/qemu-iotests/iotests.py | 8 +++ > tests/qemu-iotests/297| 44 +++ >

Re: [PATCH v4 24/34] block: Drop child_backing

2020-05-13 Thread Eric Blake
On 5/13/20 6:05 AM, Max Reitz wrote: Signed-off-by: Max Reitz --- include/block/block_int.h | 1 - block.c | 60 ++- 2 files changed, 3 insertions(+), 58 deletions(-) Reviewed-by: Eric Blake -- Eric Blake, Principal Software

Re: [PATCH v4 19/34] block: Add bdrv_default_perms()

2020-05-13 Thread Eric Blake
On 5/13/20 6:05 AM, Max Reitz wrote: This callback can be used by BDSs that use child_of_bds with the appropriate BdrvChildRole for their children. Also, make bdrv_format_default_perms() use it for child_of_bds children (just a temporary solution until we can drop bdrv_format_default_perms()

Re: [PATCH v4 23/34] block: Make backing files child_of_bds children

2020-05-13 Thread Eric Blake
On 5/13/20 6:05 AM, Max Reitz wrote: Make all parents of backing files pass the appropriate BdrvChildRole. By doing so, we can switch their BdrvChildClass over to the generic child_of_bds, which will do the right thing when given a correct BdrvChildRole. Signed-off-by: Max Reitz --- block.c

Re: [PATCH v5 15/15] floppy: make isa_fdc_get_drive_max_chs static

2020-05-13 Thread John Snow
On 5/7/20 10:06 AM, Philippe Mathieu-Daudé wrote: > On 5/7/20 3:16 PM, Gerd Hoffmann wrote: >> acpi aml generator needs this, but it is in floppy code now >> so we can make the function static. >> >> Signed-off-by: Gerd Hoffmann >> Reviewed-by: Igor Mammedov > > Reviewed-by: Philippe

Re: [PATCH v5 08/15] acpi: move aml builder code for floppy device

2020-05-13 Thread John Snow
On 5/7/20 10:05 AM, Philippe Mathieu-Daudé wrote: > +Hervé > > On 5/7/20 3:16 PM, Gerd Hoffmann wrote: >> Signed-off-by: Gerd Hoffmann >> Reviewed-by: Igor Mammedov >> --- >>   hw/block/fdc.c   | 83 oh no... >>   hw/i386/acpi-build.c | 83

Re: [PATCH v4 18/34] block: Relax *perms_for_storage for data children

2020-05-13 Thread Eric Blake
On 5/13/20 6:05 AM, Max Reitz wrote: We can be less restrictive about pure data children than those with metadata on them, so let bdrv_default_perms_for_storage() handle metadata children differently from pure data children. As explained in the code, the restrictions on metadata children are

Re: [PATCH v4 16/34] block: Pull out bdrv_default_perms_for_cow()

2020-05-13 Thread Eric Blake
On 5/13/20 6:05 AM, Max Reitz wrote: Right now, bdrv_format_default_perms() is used by format parents (generally). We want to switch to a model where most parents use a single BdrvChildClass, which then decides the permissions based on the child role. To do so, we have to split

Re: [PATCH v4 15/34] block: Distinguish paths in *_format_default_perms

2020-05-13 Thread Eric Blake
On 5/13/20 6:05 AM, Max Reitz wrote: bdrv_format_default_perms() has one code path for backing files, and one for storage files. We want to pull them out into own functions, so Grammar suggestion: s/into own/into their own/ make sure they are completely distinct before so the next patches

Re: [PATCH v4 14/34] block: Add child_of_bds

2020-05-13 Thread Eric Blake
On 5/13/20 6:05 AM, Max Reitz wrote: Any current user of child_file, child_format, and child_backing can and should use this generic BdrvChildClass instead, as it can handle all of these cases. However, to be able to do so, the users must pass the appropriate BdrvChildRole when the child is

Re: [PATCH v4 13/34] block: Unify bdrv_child_cb_detach()

2020-05-13 Thread Eric Blake
On 5/13/20 6:05 AM, Max Reitz wrote: Make bdrv_child_cb_detach() call bdrv_backing_detach() for children with a COW role (and drop the reverse call from bdrv_backing_detach()), so it can be used for any child (with a proper role set). Because so far no child has a proper role set, we need a

Re: [PATCH v4 12/34] block: Unify bdrv_child_cb_attach()

2020-05-13 Thread Eric Blake
On 5/13/20 6:05 AM, Max Reitz wrote: Make bdrv_child_cb_attach() call bdrv_backing_attach() for children with a COW role (and drop the reverse call from bdrv_backing_attach()), so it can be used for any child (with a proper role set). Because so far no child has a proper role set, we need a

Re: [PATCH] bitmaps: Add myself as maintainer

2020-05-13 Thread John Snow
On 5/13/20 10:14 AM, Eric Blake wrote: > Dirty bitmaps are important to incremental backups, including exposure > over NBD where I'm already maintainer. Also, I'm aware that lately I > have been doing as much code/review on bitmaps as John Snow, who is > hoping to scale back on this front. >

Re: [PATCH v4 11/34] block: Use bdrv_inherited_options()

2020-05-13 Thread Eric Blake
On 5/13/20 6:05 AM, Max Reitz wrote: Let child_file's, child_format's, and child_backing's .inherit_options() implementations fall back to bdrv_inherited_options() to show that it would really work for all of these cases, if only the parents passed the appropriate BdrvChildRole and

Re: [PATCH v4 10/34] block: Add generic bdrv_inherited_options()

2020-05-13 Thread Eric Blake
On 5/13/20 6:05 AM, Max Reitz wrote: After the series this patch belongs to, we want to have a common BdrvChildClass that encompasses all of child_file, child_format, and child_backing. Such a single class needs a single .inherit_options() implementation, and this patch introduces it. The next

Re: [RFC v2] migration: Add migrate-set-bitmap-node-mapping

2020-05-13 Thread Vladimir Sementsov-Ogievskiy
13.05.2020 17:56, Max Reitz wrote: This command allows mapping block node names to aliases for the purpose of block dirty bitmap migration. This way, management tools can use different node names on the source and destination and pass the mapping of how bitmaps are to be transferred to qemu (on

Re: [RFC] migration: Add migrate-set-bitmap-node-mapping

2020-05-13 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200513144941.1469447-1-mre...@redhat.com/ Hi, This series failed the docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN

Re: [RFC] migration: Add migrate-set-bitmap-node-mapping

2020-05-13 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200513144941.1469447-1-mre...@redhat.com/ Hi, This series failed the asan build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN === #!/bin/bash

  1   2   3   >