Re: [RUST] Add crate for generic vhost-user-i2c backend daemon

2021-05-27 Thread Viresh Kumar
On 28-04-21, 17:52, Viresh Kumar wrote: > Hello, > > In my earlier attempt [1], I implemented the vhost-user-i2c backend > deamon for QEMU (though the code was generic enough to be used with > any hypervisor). > > And here is a Rust implementation of the vhost-user-i2c backend > daemon. Again

Re: [PATCH v1 3/6] tests/tcg/configure.sh: tweak quoting of target_compiler

2021-05-27 Thread Thomas Huth
On 27/05/2021 18.03, Alex Bennée wrote: If you configure the host compiler with a multi-command stanza like: --cc="ccache gcc" then the configure.sh machinery falls over with confusion. Work around this by ensuring we correctly quote so where we need a complete evaluation we get it. Of

[PULL 39/44] python: add devel package requirements to setuptools

2021-05-27 Thread John Snow
setuptools doesn't have a formal understanding of development requires, but it has an optional feataures section. Fine; add a "devel" feature and add the requirements to it. To avoid duplication, we can modify pipenv to install qemu[devel] instead. This enables us to run invocations like "pip

[PULL 44/44] gitlab: add python linters to CI

2021-05-27 Thread John Snow
Add a Python container that has just enough juice for us to run the Python code quality analysis tools. Base this container on Fedora, because Fedora has very convenient packaging for testing multiple Python versions. We need python3, pip (for pulling packages), pipenv and virtualenv for creating

[PULL 42/44] python: add .gitignore

2021-05-27 Thread John Snow
Ignore *Python* build and package output (build, dist, qemu.egg-info); these files are not created as part of a QEMU build. They are created by running the commands 'python3 setup.py ' when preparing tarballs to upload to e.g. PyPI. Ignore miscellaneous cached python confetti (mypy, pylint, et

[PULL 38/44] python/qemu: add qemu package itself to pipenv

2021-05-27 Thread John Snow
This adds the python qemu packages themselves to the pipenv manifest. 'pipenv sync' will create a virtual environment sufficient to use the SDK. 'pipenv sync --dev' will create a virtual environment sufficient to use and test the SDK (with pylint, mypy, isort, flake8, etc.) The qemu packages are

[PULL 36/44] python: move .isort.cfg into setup.cfg

2021-05-27 Thread John Snow
Signed-off-by: John Snow Reviewed-by: Cleber Rosa Message-id: 20210527211715.394144-24-js...@redhat.com Signed-off-by: John Snow --- python/.isort.cfg | 7 --- python/setup.cfg | 8 2 files changed, 8 insertions(+), 7 deletions(-) delete mode 100644 python/.isort.cfg diff --git

[PULL 40/44] python: add avocado-framework and tests

2021-05-27 Thread John Snow
Try using avocado to manage our various tests; even though right now they're only invoking shell scripts and not really running any python-native code. Create tests/, and add shell scripts which call out to mypy, flake8, pylint and isort to enforce the standards in this directory. Add

[PULL 37/44] python/qemu: add isort to pipenv

2021-05-27 Thread John Snow
isort 5.0.0 through 5.0.4 has a bug that causes it to misinterpret certain "from ..." clauses that are not related to imports. isort < 5.1.1 has a bug where it does not handle comments near import statements correctly. Require 5.1.2 or greater. isort can be run (in "check" mode) with 'isort -c

[PULL 35/44] python: add mypy to pipenv

2021-05-27 Thread John Snow
0.730 appears to be about the oldest version that works with the features we want, including nice human readable output (to make sure iotest 297 passes), and type-parameterized Popen generics. 0.770, however, supports adding 'strict' to the config file, so require at least 0.770. Now that we are

[PULL 34/44] python: move mypy.ini into setup.cfg

2021-05-27 Thread John Snow
mypy supports reading its configuration values from a central project configuration file; do so. Signed-off-by: John Snow Reviewed-by: Cleber Rosa Message-id: 20210527211715.394144-22-js...@redhat.com Signed-off-by: John Snow --- python/mypy.ini | 4 python/setup.cfg | 5 + 2 files

[PULL 27/44] python: Add pipenv support

2021-05-27 Thread John Snow
pipenv is a tool used for managing virtual environments with pinned, explicit dependencies. It is used for precisely recreating python virtual environments. pipenv uses two files to do this: (1) Pipfile, which is similar in purpose and scope to what setup.cfg lists. It specifies the requisite

[PULL 30/44] python: add pylint to pipenv

2021-05-27 Thread John Snow
We are specifying >= pylint 2.8.x for several reasons: 1. For setup.cfg support, added in pylint 2.5.x 2. To specify a version that has incompatibly dropped bad-whitespace checks (2.6.x) 3. 2.7.x fixes "unsubscriptable" warnings in Python 3.9 4. 2.8.x adds a new, incompatible

[PULL 24/44] python: add VERSION file

2021-05-27 Thread John Snow
Python infrastructure as it exists today is not capable reliably of single-sourcing a package version from a parent directory. The authors of pip are working to correct this, but as of today this is not possible. The problem is that when using pip to build and install a python package, it copies

[PULL 31/44] python: move flake8 config to setup.cfg

2021-05-27 Thread John Snow
Update the comment concerning the flake8 exception to match commit 42c0dd12, whose commit message stated: A note on the flake8 exception: flake8 will warn on *any* bare except, but pylint's is context-aware and will suppress the warning if you re-raise the exception. Signed-off-by: John Snow

[PULL 25/44] python: add directory structure README.rst files

2021-05-27 Thread John Snow
Add short readmes to python/, python/qemu/, python/qemu/machine, python/qemu/qmp, and python/qemu/utils that explain the directory hierarchy. These readmes are visible when browsing the source on e.g. gitlab/github and are designed to help new developers/users quickly make sense of the source

[PULL 29/44] python: move pylintrc into setup.cfg

2021-05-27 Thread John Snow
Delete the empty settings now that it's sharing a home with settings for other tools. pylint can now be run from this folder as "pylint qemu". Signed-off-by: John Snow Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa Message-id: 20210527211715.394144-17-js...@redhat.com Signed-off-by: John

[PULL 26/44] python: add MANIFEST.in

2021-05-27 Thread John Snow
When creating a source or binary distribution via 'python3 setup.py ', the VERSION and PACKAGE.rst files aren't bundled by default. Create a MANIFEST.in file that instructs the build tools to include these so that installation from these files won't fail. This is required by 'tox', as well as by

[PULL 22/44] python: create qemu packages

2021-05-27 Thread John Snow
move python/qemu/*.py to python/qemu/[machine, qmp, utils]/*.py and update import directives across the tree. This is done to create a PEP420 namespace package, in which we may create subpackages. To do this, the namespace directory ("qemu") should not have any modules in it. Those files will go

[PULL 43/44] python: add tox support

2021-05-27 Thread John Snow
This is intended to be a manually run, non-CI script. Use tox to test the linters against all python versions from 3.6 to 3.10. This will only work if you actually have those versions installed locally, but Fedora makes this easy: > sudo dnf install python3.6 python3.7 python3.8 python3.9

[PULL 19/44] python/machine: disable warning for Popen in _launch()

2021-05-27 Thread John Snow
We handle this resource rather meticulously in shutdown/kill/wait/__exit__ et al, through the laborious mechanisms in _do_shutdown(). Quiet this pylint warning here. Signed-off-by: John Snow Reviewed-by: Cleber Rosa Message-id: 20210527211715.394144-7-js...@redhat.com Message-id:

[PULL 23/44] python: add qemu package installer

2021-05-27 Thread John Snow
Add setup.cfg and setup.py, necessary for installing a package via pip. Add a ReST document (PACKAGE.rst) explaining the basics of what this package is for and who to contact for more information. This document will be used as the landing page for the package on PyPI. List the subpackages we

[PULL 20/44] python/machine: Trim line length to below 80 chars

2021-05-27 Thread John Snow
One more little delinting fix that snuck in. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Cleber Rosa Message-id: 20210527211715.394144-8-js...@redhat.com Signed-off-by: John Snow --- python/qemu/machine.py | 2 +- 1 file changed, 1 insertion(+), 1

[PULL 41/44] python: add Makefile for some common tasks

2021-05-27 Thread John Snow
Add "make venv" to create the pipenv-managed virtual environment that contains our explicitly pinned dependencies. Add "make check" to run the python linters [in the host execution environment]. Add "make venv-check" which combines the above two: create/update the venv, then run the linters in

[PULL 21/44] iotests/297: add --namespace-packages to mypy arguments

2021-05-27 Thread John Snow
mypy is kind of weird about how it handles imports. For legacy reasons, it won't load PEP 420 namespaces, because of logic implemented prior to that becoming a standard. So, if you plan on using any, you have to pass --namespace-packages. Alright, fine. Signed-off-by: John Snow Reviewed-by:

[PULL 14/44] python/console_socket: avoid one-letter variable

2021-05-27 Thread John Snow
Fixes pylint warnings. Signed-off-by: John Snow Reviewed-by: Cleber Rosa Reviewed-by: Philippe Mathieu-Daudé Message-id: 20210527211715.394144-2-js...@redhat.com Message-id: 20210517184808.3562549-2-js...@redhat.com Signed-off-by: John Snow --- python/qemu/console_socket.py | 10 +-

[PULL 16/44] python/machine: use subprocess.run instead of subprocess.Popen

2021-05-27 Thread John Snow
use run() instead of Popen() -- to assert to pylint that we are not forgetting to close a long-running program. Signed-off-by: John Snow Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa Message-id: 20210527211715.394144-4-js...@redhat.com Message-id: 20210517184808.3562549-4-js...@redhat.com

[PULL 33/44] python: Add flake8 to pipenv

2021-05-27 Thread John Snow
flake8 3.5.x does not support the --extend-ignore syntax used in the .flake8 file to gracefully extend default ignores, so 3.6.x is our minimum requirement. There is no known upper bound. flake8 can be run from the python/ directory with no arguments. Signed-off-by: John Snow Reviewed-by:

[PULL 13/44] acceptance tests: bump Avocado version to 88.1

2021-05-27 Thread John Snow
From: Willian Rampazzo Besides some internal changes, new features, and bug fixes, on the QEMU side, this version fixes the following message seen when running the acceptance tests: "Error running method "pre_tests" of plugin "fetchasset": 'bytes' object has no attribute 'encode'". The release

[PULL 18/44] python/machine: Disable pylint warning for open() in _pre_launch

2021-05-27 Thread John Snow
Shift the open() call later so that the pylint pragma applies *only* to that one open() call. Add a note that suggests why this is safe: the resource is unconditionally cleaned up in _post_shutdown(). _post_shutdown is called after failed launches (see launch()), and unconditionally after every

[PULL 32/44] python: add excluded dirs to flake8 config

2021-05-27 Thread John Snow
Instruct flake8 to avoid certain well-known directories created by python tooling that it ought not check. Note that at-present, nothing actually creates a ".venv" directory; but it is in such widespread usage as a de-facto location for a developer's virtual environment that it should be excluded

[PULL 11/44] Acceptance Tests: introduce CPU hotplug test

2021-05-27 Thread John Snow
From: Cleber Rosa Even though there are qtest based tests for hotplugging CPUs (from which this test took some inspiration from), this one adds checks from a Linux guest point of view. It should also serve as an example for tests that follow a similar pattern and need to interact with QEMU (via

[PULL 17/44] python/console_socket: Add a pylint ignore

2021-05-27 Thread John Snow
We manage cleaning up this resource ourselves. Pylint should shush. Signed-off-by: John Snow Reviewed-by: Cleber Rosa Message-id: 20210527211715.394144-5-js...@redhat.com Message-id: 20210517184808.3562549-5-js...@redhat.com Signed-off-by: John Snow --- python/qemu/console_socket.py | 1 + 1

[PULL 12/44] tests/acceptance/virtiofs_submounts.py: fix setup of SSH pubkey

2021-05-27 Thread John Snow
From: Cleber Rosa The public key argument should be a path to a file, and not the public key data. Reported-by: Wainer dos Santos Moschetta Signed-off-by: Cleber Rosa Message-Id: <20210412044644.55083-12-cr...@redhat.com> Reviewed-by: Willian Rampazzo Reviewed-by: Wainer dos Santos Moschetta

[PULL 08/44] Acceptance Tests: set up SSH connection by default after boot for LinuxTest

2021-05-27 Thread John Snow
From: Cleber Rosa The LinuxTest specifically targets users that need to interact with Linux guests. So, it makes sense to give a connection by default, and avoid requiring it as boiler-plate code. Signed-off-by: Cleber Rosa Reviewed-by: Marc-André Lureau Reviewed-by: Willian Rampazzo

[PULL 28/44] python: add pylint import exceptions

2021-05-27 Thread John Snow
Pylint 2.5.x - 2.7.x have regressions that make import checking inconsistent, see: https://github.com/PyCQA/pylint/issues/3609 https://github.com/PyCQA/pylint/issues/3624 https://github.com/PyCQA/pylint/issues/3651 Pinning to 2.4.4 is worse, because it mandates versions of shared dependencies

[PULL 15/44] python/machine: use subprocess.DEVNULL instead of open(os.path.devnull)

2021-05-27 Thread John Snow
One less file resource to manage, and it helps quiet some pylint >= 2.8.0 warnings about not using a with-context manager for the open call. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Cleber Rosa Message-id: 20210527211715.394144-3-js...@redhat.com Message-id:

[PULL 09/44] tests/acceptance/virtiofs_submounts.py: remove launch_vm()

2021-05-27 Thread John Snow
From: Cleber Rosa The LinuxTest class' launch_and_wait() method now behaves the same way as this test's custom launch_vm(), so let's just use the upper layer (common) method. Signed-off-by: Cleber Rosa Reviewed-by: Marc-André Lureau Reviewed-by: Eric Auger Reviewed-by: Willian Rampazzo

[PULL 02/44] tests/acceptance/virtiofs_submounts.py: add missing accel tag

2021-05-27 Thread John Snow
From: Cleber Rosa The tag is useful to select tests that depend/use a particular feature. Signed-off-by: Cleber Rosa Reviewed-by: Wainer dos Santos Moschetta Reviewed-by: Willian Rampazzo Reviewed-by: Eric Auger Message-Id: <20210412044644.55083-2-cr...@redhat.com> Signed-off-by: John Snow

[PULL 10/44] Acceptance Tests: add basic documentation on LinuxTest base class

2021-05-27 Thread John Snow
From: Cleber Rosa Signed-off-by: Cleber Rosa Reviewed-by: Marc-André Lureau Reviewed-by: Willian Rampazzo Reviewed-by: Eric Auger Reviewed-by: Wainer dos Santos Moschetta Message-Id: <20210412044644.55083-10-cr...@redhat.com> Signed-off-by: John Snow --- docs/devel/testing.rst | 26

[PULL 07/44] Acceptance Tests: make username/password configurable

2021-05-27 Thread John Snow
From: Cleber Rosa This makes the username/password used for authentication configurable, because some guest operating systems may have restrictions on accounts to be used for logins, and it just makes it better documented. Signed-off-by: Cleber Rosa Reviewed-by: Marc-André Lureau Reviewed-by:

[PULL 06/44] Acceptance Tests: add port redirection for ssh by default

2021-05-27 Thread John Snow
From: Cleber Rosa For users of the LinuxTest class, let's set up the VM with the port redirection for SSH, instead of requiring each test to set the same arguments. It also sets the network device, by default, to virtio-net. Signed-off-by: Cleber Rosa Reviewed-by: Marc-André Lureau

[PULL 00/44] Python patches

2021-05-27 Thread John Snow
The following changes since commit c8616fc7670b884de5f74d2767aade224c1c5c3a: Merge remote-tracking branch 'remotes/philmd/tags/gitlab-ci-20210527' into staging (2021-05-27 16:32:57 +0100) are available in the Git repository at: https://gitlab.com/jsnow/qemu.git tags/python-pull-request

[PULL 05/44] Acceptance Tests: move useful ssh methods to base class

2021-05-27 Thread John Snow
From: Cleber Rosa Both the virtiofs submounts and the linux ssh mips malta tests contains useful methods related to ssh that deserve to be made available to other tests. Let's move them to an auxiliary, mix-in class that will be used on the base LinuxTest class. The method that helps with

[PULL 03/44] tests/acceptance/virtiofs_submounts.py: evaluate string not length

2021-05-27 Thread John Snow
From: Cleber Rosa If the vmlinuz variable is set to anything that evaluates to True, then the respective arguments should be set. If the variable contains an empty string, than it will evaluate to False, and the extra arguments will not be set. This keeps the same logic, but improves

[PULL 04/44] Python: add utility function for retrieving port redirection

2021-05-27 Thread John Snow
From: Cleber Rosa Slightly different versions for the same utility code are currently present on different locations. This unifies them all, giving preference to the version from virtiofs_submounts.py, because of the last tweaks added to it. While at it, this adds a "qemu.utils" module to host

[PULL 01/44] Python: expose QEMUMachine's temporary directory

2021-05-27 Thread John Snow
From: Cleber Rosa Each instance of qemu.machine.QEMUMachine currently has a "test directory", which may not have any relation to a "test", and it's really a temporary directory. Users instantiating the QEMUMachine class will be able to set the location of the directory that will *contain* the

Re: [PATCH] vfio: Fix unregister SaveVMHandler in vfio_migration_finalize

2021-05-27 Thread Kunkun Jiang
Hi Philippe, On 2021/5/27 21:44, Philippe Mathieu-Daudé wrote: On 5/27/21 2:31 PM, Kunkun Jiang wrote: In the vfio_migration_init(), the SaveVMHandler is registered for VFIO device. But it lacks the operation of 'unregister'. It will lead to 'Segmentation fault (core dumped)' in

Re: [PATCH] target/riscv: hardwire bits in hideleg and hedeleg

2021-05-27 Thread LIU Zhiwei
Reviewed-by: LIU Zhiwei Zhiwei On 5/22/21 11:59 PM, Jose Martins wrote: The specification mandates for certain bits to be hardwired in the hypervisor delegation registers. This was not being enforced. Signed-off-by: Jose Martins --- target/riscv/csr.c | 13 +++-- 1 file changed,

Re: [PATCH] target/riscv: hardwire bits in hideleg and hedeleg

2021-05-27 Thread LIU Zhiwei
On 5/26/21 1:50 AM, Jose Martins wrote: We can use it directly if only one macro VS_MODE_INTERRUPTS. I wrote it like this to be more coherent with what was already there which also makes it more readable. Furthermore, the compiler will just probably optimize the variable away, right? Hi

Re: [PATCH v2] Update libslirp to v4.5.0

2021-05-27 Thread Doug Evans
On Wed, May 19, 2021 at 8:18 AM wrote: > From: Marc-André Lureau > > Switch from stable-4.2 branch to upstream v4.5.0 release. > > ## [4.5.0] - 2021-05-18 > > ### Added > > - IPv6 forwarding. !62 !75 !77 > - slirp_neighbor_info() to dump the ARP/NDP tables. !71 > > ### Changed > > - Lazy

Re: [PATCH v3] target/riscv: fix VS interrupts forwarding to HS

2021-05-27 Thread LIU Zhiwei
On 5/28/21 6:34 AM, Alistair Francis wrote: On Sun, May 23, 2021 at 1:45 AM Jose Martins wrote: VS interrupts (2, 6, 10) were not correctly forwarded to hs-mode when not delegated in hideleg (which was not being taken into account). This was mainly because hs level sie was not always

Re: [PATCH 1/1] target/riscv: Fix vsip vsie CSR ops in M and HS mode

2021-05-27 Thread LIU Zhiwei
On 5/28/21 6:19 AM, Alistair Francis wrote: On Thu, May 27, 2021 at 7:01 PM LIU Zhiwei wrote: When V=1, instructions that normally read or modify a supervisor CSR shall instead access the corresponding VS CSR. And the VS CSRs can be accessed as themselves from M-mode or HS-mode. In M and HS

Performance issue with qcow2/raid

2021-05-27 Thread Jose R. Ziviani
Hello team, I'm currently investigating a performance regression detected by iozone filesystem benchmark (https://www.iozone.org/). Basically, if I format a QCOW2 image with XFS filesystem in my guest and run iozone I'll get the following result: $ mkfs.xfs -f /dev/xvdb1 && \ mount -t xfs

Re: [PATCH v3] target/riscv: fix VS interrupts forwarding to HS

2021-05-27 Thread Alistair Francis
On Sun, May 23, 2021 at 1:45 AM Jose Martins wrote: > > VS interrupts (2, 6, 10) were not correctly forwarded to hs-mode when > not delegated in hideleg (which was not being taken into account). This > was mainly because hs level sie was not always considered enabled when > it should. The spec

Re: [PATCH 1/1] target/riscv: Fix vsip vsie CSR ops in M and HS mode

2021-05-27 Thread Alistair Francis
On Thu, May 27, 2021 at 7:01 PM LIU Zhiwei wrote: > > When V=1, instructions that normally read or modify a supervisor CSR > shall instead access the corresponding VS CSR. And the VS CSRs can be > accessed as themselves from M-mode or HS-mode. > > In M and HS mode, VSIP or VSIE should be written

Re: [PATCH v6 00/17] support subsets of bitmanip extension

2021-05-27 Thread Alistair Francis
On Thu, May 6, 2021 at 2:11 AM wrote: > > From: Frank Chang > > This patchset implements RISC-V B-extension v0.93 version Zba, Zbb and > Zbs subset instructions. Some Zbp instructions are also implemented as > they have similar behavior with their Zba-, Zbb- and Zbs-family > instructions or for

Re: [PATCH v6 17/17] target/riscv: rvb: add b-ext version cpu option

2021-05-27 Thread Alistair Francis
On Thu, May 6, 2021 at 2:27 AM wrote: > > From: Frank Chang > > Default b-ext version is v0.93. > > Signed-off-by: Frank Chang Reviewed-by: Alistair Francis Alistair > --- > target/riscv/cpu.c | 23 +++ > target/riscv/cpu.h | 3 +++ > 2 files changed, 26 insertions(+)

Re: [PATCH 0/2] [RESEND] SEV firmware error list touchups

2021-05-27 Thread Philippe Mathieu-Daudé
ping^3... On Tue, May 11, 2021 at 9:35 AM Philippe Mathieu-Daudé wrote: > > Cc'ing qemu-trivial@ > > On 4/30/21 3:48 PM, Connor Kuehl wrote: > > Connor Kuehl (2): > > sev: use explicit indices for mapping firmware error codes to strings > > sev: add missing firmware error conditions > > > >

[PATCH v8 31/31] gitlab: add python linters to CI

2021-05-27 Thread John Snow
Add a Python container that has just enough juice for us to run the Python code quality analysis tools. Base this container on Fedora, because Fedora has very convenient packaging for testing multiple Python versions. We need python3, pip (for pulling packages), pipenv and virtualenv for creating

[PATCH v8 26/31] python: add devel package requirements to setuptools

2021-05-27 Thread John Snow
setuptools doesn't have a formal understanding of development requires, but it has an optional feataures section. Fine; add a "devel" feature and add the requirements to it. To avoid duplication, we can modify pipenv to install qemu[devel] instead. This enables us to run invocations like "pip

[PATCH v8 29/31] python: add .gitignore

2021-05-27 Thread John Snow
Ignore *Python* build and package output (build, dist, qemu.egg-info); these files are not created as part of a QEMU build. They are created by running the commands 'python3 setup.py ' when preparing tarballs to upload to e.g. PyPI. Ignore miscellaneous cached python confetti (mypy, pylint, et

[PATCH v8 24/31] python/qemu: add isort to pipenv

2021-05-27 Thread John Snow
isort 5.0.0 through 5.0.4 has a bug that causes it to misinterpret certain "from ..." clauses that are not related to imports. isort < 5.1.1 has a bug where it does not handle comments near import statements correctly. Require 5.1.2 or greater. isort can be run (in "check" mode) with 'isort -c

[PATCH v8 22/31] python: add mypy to pipenv

2021-05-27 Thread John Snow
0.730 appears to be about the oldest version that works with the features we want, including nice human readable output (to make sure iotest 297 passes), and type-parameterized Popen generics. 0.770, however, supports adding 'strict' to the config file, so require at least 0.770. Now that we are

[PATCH v8 23/31] python: move .isort.cfg into setup.cfg

2021-05-27 Thread John Snow
Signed-off-by: John Snow Reviewed-by: Cleber Rosa --- python/.isort.cfg | 7 --- python/setup.cfg | 8 2 files changed, 8 insertions(+), 7 deletions(-) delete mode 100644 python/.isort.cfg diff --git a/python/.isort.cfg b/python/.isort.cfg deleted file mode 100644 index

[PATCH v8 30/31] python: add tox support

2021-05-27 Thread John Snow
This is intended to be a manually run, non-CI script. Use tox to test the linters against all python versions from 3.6 to 3.10. This will only work if you actually have those versions installed locally, but Fedora makes this easy: > sudo dnf install python3.6 python3.7 python3.8 python3.9

[PATCH v8 20/31] python: Add flake8 to pipenv

2021-05-27 Thread John Snow
flake8 3.5.x does not support the --extend-ignore syntax used in the .flake8 file to gracefully extend default ignores, so 3.6.x is our minimum requirement. There is no known upper bound. flake8 can be run from the python/ directory with no arguments. Signed-off-by: John Snow Reviewed-by:

[PATCH v8 25/31] python/qemu: add qemu package itself to pipenv

2021-05-27 Thread John Snow
This adds the python qemu packages themselves to the pipenv manifest. 'pipenv sync' will create a virtual environment sufficient to use the SDK. 'pipenv sync --dev' will create a virtual environment sufficient to use and test the SDK (with pylint, mypy, isort, flake8, etc.) The qemu packages are

[PATCH v8 21/31] python: move mypy.ini into setup.cfg

2021-05-27 Thread John Snow
mypy supports reading its configuration values from a central project configuration file; do so. Signed-off-by: John Snow Reviewed-by: Cleber Rosa --- python/mypy.ini | 4 python/setup.cfg | 5 + 2 files changed, 5 insertions(+), 4 deletions(-) delete mode 100644 python/mypy.ini

[PATCH v8 28/31] python: add Makefile for some common tasks

2021-05-27 Thread John Snow
Add "make venv" to create the pipenv-managed virtual environment that contains our explicitly pinned dependencies. Add "make check" to run the python linters [in the host execution environment]. Add "make venv-check" which combines the above two: create/update the venv, then run the linters in

[PATCH v8 17/31] python: add pylint to pipenv

2021-05-27 Thread John Snow
We are specifying >= pylint 2.8.x for several reasons: 1. For setup.cfg support, added in pylint 2.5.x 2. To specify a version that has incompatibly dropped bad-whitespace checks (2.6.x) 3. 2.7.x fixes "unsubscriptable" warnings in Python 3.9 4. 2.8.x adds a new, incompatible

Re: [PATCH v4] i386: Add ratelimit for bus locks acquired in guest

2021-05-27 Thread Eduardo Habkost
On Fri, May 21, 2021 at 12:38:20PM +0800, Chenyi Qiang wrote: [...] > @@ -4222,6 +4247,15 @@ void kvm_arch_pre_run(CPUState *cpu, struct kvm_run > *run) > } > } > > +static void kvm_rate_limit_on_bus_lock(void) > +{ > +uint64_t delay_ns =

[PATCH v8 19/31] python: add excluded dirs to flake8 config

2021-05-27 Thread John Snow
Instruct flake8 to avoid certain well-known directories created by python tooling that it ought not check. Note that at-present, nothing actually creates a ".venv" directory; but it is in such widespread usage as a de-facto location for a developer's virtual environment that it should be excluded

[PATCH v8 27/31] python: add avocado-framework and tests

2021-05-27 Thread John Snow
Try using avocado to manage our various tests; even though right now they're only invoking shell scripts and not really running any python-native code. Create tests/, and add shell scripts which call out to mypy, flake8, pylint and isort to enforce the standards in this directory. Add

[PATCH v8 14/31] python: Add pipenv support

2021-05-27 Thread John Snow
pipenv is a tool used for managing virtual environments with pinned, explicit dependencies. It is used for precisely recreating python virtual environments. pipenv uses two files to do this: (1) Pipfile, which is similar in purpose and scope to what setup.cfg lists. It specifies the requisite

[PATCH v8 13/31] python: add MANIFEST.in

2021-05-27 Thread John Snow
When creating a source or binary distribution via 'python3 setup.py ', the VERSION and PACKAGE.rst files aren't bundled by default. Create a MANIFEST.in file that instructs the build tools to include these so that installation from these files won't fail. This is required by 'tox', as well as by

[PATCH v8 18/31] python: move flake8 config to setup.cfg

2021-05-27 Thread John Snow
Update the comment concerning the flake8 exception to match commit 42c0dd12, whose commit message stated: A note on the flake8 exception: flake8 will warn on *any* bare except, but pylint's is context-aware and will suppress the warning if you re-raise the exception. Signed-off-by: John Snow

[PATCH v8 16/31] python: move pylintrc into setup.cfg

2021-05-27 Thread John Snow
Delete the empty settings now that it's sharing a home with settings for other tools. pylint can now be run from this folder as "pylint qemu". Signed-off-by: John Snow Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa --- python/qemu/machine/pylintrc | 58

[PATCH v8 09/31] python: create qemu packages

2021-05-27 Thread John Snow
move python/qemu/*.py to python/qemu/[machine, qmp, utils]/*.py and update import directives across the tree. This is done to create a PEP420 namespace package, in which we may create subpackages. To do this, the namespace directory ("qemu") should not have any modules in it. Those files will go

[PATCH v8 05/31] python/machine: Disable pylint warning for open() in _pre_launch

2021-05-27 Thread John Snow
Shift the open() call later so that the pylint pragma applies *only* to that one open() call. Add a note that suggests why this is safe: the resource is unconditionally cleaned up in _post_shutdown(). _post_shutdown is called after failed launches (see launch()), and unconditionally after every

[PATCH v8 15/31] python: add pylint import exceptions

2021-05-27 Thread John Snow
Pylint 2.5.x - 2.7.x have regressions that make import checking inconsistent, see: https://github.com/PyCQA/pylint/issues/3609 https://github.com/PyCQA/pylint/issues/3624 https://github.com/PyCQA/pylint/issues/3651 Pinning to 2.4.4 is worse, because it mandates versions of shared dependencies

[PATCH v8 12/31] python: add directory structure README.rst files

2021-05-27 Thread John Snow
Add short readmes to python/, python/qemu/, python/qemu/machine, python/qemu/qmp, and python/qemu/utils that explain the directory hierarchy. These readmes are visible when browsing the source on e.g. gitlab/github and are designed to help new developers/users quickly make sense of the source

[PATCH v8 07/31] python/machine: Trim line length to below 80 chars

2021-05-27 Thread John Snow
One more little delinting fix that snuck in. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Cleber Rosa --- python/qemu/machine.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index

[PATCH v8 04/31] python/console_socket: Add a pylint ignore

2021-05-27 Thread John Snow
We manage cleaning up this resource ourselves. Pylint should shush. Signed-off-by: John Snow Reviewed-by: Cleber Rosa Message-id: 20210517184808.3562549-5-js...@redhat.com Signed-off-by: John Snow --- python/qemu/console_socket.py | 1 + 1 file changed, 1 insertion(+) diff --git

[PATCH v8 11/31] python: add VERSION file

2021-05-27 Thread John Snow
Python infrastructure as it exists today is not capable reliably of single-sourcing a package version from a parent directory. The authors of pip are working to correct this, but as of today this is not possible. The problem is that when using pip to build and install a python package, it copies

[PATCH v8 06/31] python/machine: disable warning for Popen in _launch()

2021-05-27 Thread John Snow
We handle this resource rather meticulously in shutdown/kill/wait/__exit__ et al, through the laborious mechanisms in _do_shutdown(). Quiet this pylint warning here. Signed-off-by: John Snow Reviewed-by: Cleber Rosa Message-id: 20210517184808.3562549-7-js...@redhat.com Signed-off-by: John Snow

[PATCH v8 08/31] iotests/297: add --namespace-packages to mypy arguments

2021-05-27 Thread John Snow
mypy is kind of weird about how it handles imports. For legacy reasons, it won't load PEP 420 namespaces, because of logic implemented prior to that becoming a standard. So, if you plan on using any, you have to pass --namespace-packages. Alright, fine. Signed-off-by: John Snow Reviewed-by:

[PATCH v8 01/31] python/console_socket: avoid one-letter variable

2021-05-27 Thread John Snow
Fixes pylint warnings. Signed-off-by: John Snow Reviewed-by: Cleber Rosa Reviewed-by: Philippe Mathieu-Daudé Message-id: 20210517184808.3562549-2-js...@redhat.com Signed-off-by: John Snow --- python/qemu/console_socket.py | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff

[PATCH v8 10/31] python: add qemu package installer

2021-05-27 Thread John Snow
Add setup.cfg and setup.py, necessary for installing a package via pip. Add a ReST document (PACKAGE.rst) explaining the basics of what this package is for and who to contact for more information. This document will be used as the landing page for the package on PyPI. List the subpackages we

[PATCH v8 03/31] python/machine: use subprocess.run instead of subprocess.Popen

2021-05-27 Thread John Snow
use run() instead of Popen() -- to assert to pylint that we are not forgetting to close a long-running program. Signed-off-by: John Snow Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa Message-id: 20210517184808.3562549-4-js...@redhat.com Signed-off-by: John Snow --- python/qemu/machine.py |

[PATCH v8 02/31] python/machine: use subprocess.DEVNULL instead of open(os.path.devnull)

2021-05-27 Thread John Snow
One less file resource to manage, and it helps quiet some pylint >= 2.8.0 warnings about not using a with-context manager for the open call. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Cleber Rosa Message-id: 20210517184808.3562549-3-js...@redhat.com

[PATCH v8 00/31] python: create installable package

2021-05-27 Thread John Snow
Based-on: https://gitlab.com/cleber.gnu/qemu/-/commits/python-next CI: https://gitlab.com/jsnow/qemu/-/pipelines/310845984 GitLab: https://gitlab.com/jsnow/qemu/-/tree/python-package-mk4 MR: https://gitlab.com/jsnow/qemu/-/merge_requests/7 (Note: v8 is just being sent to collate final RB/TB tags

Re: [PATCH v4 1/2] target/i386: Trivial code motion

2021-05-27 Thread Eduardo Habkost
On Tue, May 18, 2021 at 10:53:51AM +0800, Ziqiao Kong wrote: > On Tue, May 18, 2021 at 4:16 AM Eduardo Habkost wrote: > > > > On Fri, May 07, 2021 at 04:00:56PM +0800, Ziqiao Kong wrote: > > > Move the float translation case to a new block by a new pair of braces. > > > > If you are just adding

Re: [PATCH v4 2/2] target/i386: Correct implementation for FCS, FIP, FDS and FDP

2021-05-27 Thread Eduardo Habkost
On Tue, May 18, 2021 at 11:06:56AM +0800, Ziqiao Kong wrote: [...] > > > +/* > > > + * If CR0.PE = 1, each instruction saves FCS and FDS into memory. If > > > + * CPUID.(EAX=07H,ECX=0H):EBX[bit 13] = 1, the processor deprecates > > > + * FCS and FDS; it saves each as H. > > > +

Re: [PATCH 5/5] hw/core/cpu: removed cpu_dump_statistics function

2021-05-27 Thread Eduardo Habkost
On Wed, May 26, 2021 at 05:21:04PM -0300, Bruno Larsen (billionai) wrote: > No more architectures set the pointer to dump_statistics, so there's no > point in keeping it, or the related cpu_dump_statistics function. > > Suggested-by: Richard Henderson > Signed-off-by: Bruno Larsen (billionai)

Re: [PATCH v3] target/i386/sev: add support to query the attestation report

2021-05-27 Thread Eduardo Habkost
On Thu, Apr 29, 2021 at 12:07:28PM -0500, Brijesh Singh wrote: > The SEV FW >= 0.23 added a new command that can be used to query the > attestation report containing the SHA-256 digest of the guest memory > and VMSA encrypted with the LAUNCH_UPDATE and sign it with the PEK. > > Note, we already

Re: [PATCH 2/2] tests/acceptance: Add tests for the Pegasos2 machine

2021-05-27 Thread Wainer dos Santos Moschetta
Hi, On 5/15/21 10:45 AM, Philippe Mathieu-Daudé wrote: Add a pair of tests for the Pegasos2 machine following the steps from: https://lists.nongnu.org/archive/html/qemu-devel/2021-01/msg01553.html $ PEGASOS2_ROM_PATH=/tmp/pegasos2.rom AVOCADO_ALLOW_UNTRUSTED_CODE=1 \ avocado

Re: [PATCH] tcg/ppc: Fix building with Clang

2021-05-27 Thread Brad Smith
On 5/2/2021 12:02 AM, Brad Smith wrote: On 4/22/2021 11:39 AM, Richard Henderson wrote: On 4/22/21 2:20 AM, Peter Maydell wrote: On Thu, 22 Apr 2021 at 06:18, Richard Henderson I'm thinking something like #if !defined(_CALL_SYSV) && \   !defined(_CALL_DARWIN) && \  

Re: [PATCH] arm: Consistently use "Cortex-Axx", not "Cortex Axx"

2021-05-27 Thread Richard Henderson
On 5/27/21 2:51 AM, Peter Maydell wrote: The official punctuation for Arm CPU names uses a hyphen, like "Cortex-A9". We mostly follow this, but in a few places usage without the hyphen has crept in. Fix those so we consistently use the same way of writing the CPU name. This commit was created

Re: [PATCH v2 2/6] file-posix: try BLKSECTGET on block devices too, do not round to power of 2

2021-05-27 Thread Paolo Bonzini
On 27/05/21 17:51, Kevin Wolf wrote: Am 24.05.2021 um 18:36 hat Paolo Bonzini geschrieben: bs->sg is only true for character devices, but block devices can also be used with scsi-block and scsi-generic. Unfortunately BLKSECTGET returns bytes in an int for /dev/sgN devices, and sectors in a

  1   2   3   4   >