Add a python container that contains just enough juice for us to run the python
code quality analysis tools.
Base this container on fedora, because fedora has very convenient
packaging for testing multiple python versions.
Add two tests:
check-python-pipenv uses pipenv to test a frozen, very
setuptools doesn't have a formal understanding of development requires,
but it has an optional feataures section. Fine; add a "devel" feature
and add the requirements to it.
To avoid duplication, we can modify pipenv to install qemu[devel]
instead. This enables us to run invocations like "pip
This is intended to be a manually run, non-CI script.
Use tox to test the linters against all python versions from 3.6 to
3.10. This will only work if you actually have those versions installed
locally, but Fedora makes this easy:
> sudo dnf install python3.6 python3.7 python3.8 python3.9
Ignore *Python* build and package output (build, dist, qemu.egg-info);
these files are not created as part of a QEMU build. They are created by
running the commands 'python3 setup.py ' when preparing
tarballs to upload to e.g. PyPI.
Ignore miscellaneous cached python confetti (mypy, pylint, et
This adds the python qemu packages themselves to the pipenv manifest.
'pipenv sync' will create a virtual environment sufficient to use the SDK.
'pipenv sync --dev' will create a virtual environment sufficient to use
and test the SDK (with pylint, mypy, isort, flake8, etc.)
The qemu packages are
isort 5.0.0 through 5.0.4 has a bug that causes it to misinterpret
certain "from ..." clauses that are not related to imports.
isort < 5.1.1 has a bug where it does not handle comments near import
statements correctly.
Require 5.1.2 or greater.
isort can be run (in "check" mode) with 'isort -c
0.730 appears to be about the oldest version that works with the
features we want, including nice human readable output (to make sure
iotest 297 passes), and type-parameterized Popen generics.
0.770, however, supports adding 'strict' to the config file, so require
at least 0.770.
Now that we are
Update the comment concerning the flake8 exception to match commit
42c0dd12, whose commit message stated:
A note on the flake8 exception: flake8 will warn on *any* bare except,
but pylint's is context-aware and will suppress the warning if you
re-raise the exception.
Signed-off-by: John Snow
Instruct flake8 to avoid certain well-known directories created by
python tooling that it ought not check.
Note that at-present, nothing actually creates a ".venv" directory; but
it is in such widespread usage as a de-facto location for a developer's
virtual environment that it should be excluded
Delete the empty settings now that it's sharing a home with settings for
other tools.
pylint can now be run from this folder as "pylint qemu".
Signed-off-by: John Snow
Reviewed-by: Cleber Rosa
Tested-by: Cleber Rosa
---
python/qemu/machine/pylintrc | 58
Add "make venv" to create the pipenv-managed virtual environment that
contains our explicitly pinned dependencies.
Add "make check" to run the python linters [in the host execution
environment].
Add "make venv-check" which combines the above two: create/update the
venv, then run the linters in
Try using avocado to manage our various tests; even though right now
they're only invoking shell scripts and not really running any
python-native code.
Create tests/, and add shell scripts which call out to mypy, flake8,
pylint and isort to enforce the standards in this directory.
Add
Signed-off-by: John Snow
Reviewed-by: Cleber Rosa
---
python/.isort.cfg | 7 ---
python/setup.cfg | 8
2 files changed, 8 insertions(+), 7 deletions(-)
delete mode 100644 python/.isort.cfg
diff --git a/python/.isort.cfg b/python/.isort.cfg
deleted file mode 100644
index
pipenv is a tool used for managing virtual environments with pinned,
explicit dependencies. It is used for precisely recreating python
virtual environments.
pipenv uses two files to do this:
(1) Pipfile, which is similar in purpose and scope to what setup.cfg
lists. It specifies the requisite
One more little delinting fix that snuck in.
Signed-off-by: John Snow
---
python/qemu/machine.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index 5d72c4ca369..a8837b36e47 100644
--- a/python/qemu/machine.py
+++
flake8 3.5.x does not support the --extend-ignore syntax used in the
.flake8 file to gracefully extend default ignores, so 3.6.x is our
minimum requirement. There is no known upper bound.
flake8 can be run from the python/ directory with no arguments.
Signed-off-by: John Snow
Reviewed-by:
move python/qemu/*.py to python/qemu/[machine, qmp, utils]/*.py and
update import directives across the tree.
This is done to create a PEP420 namespace package, in which we may
create subpackages. To do this, the namespace directory ("qemu") should
not have any modules in it. Those files will go
mypy supports reading its configuration values from a central project
configuration file; do so.
Signed-off-by: John Snow
Reviewed-by: Cleber Rosa
---
python/mypy.ini | 4
python/setup.cfg | 5 +
2 files changed, 5 insertions(+), 4 deletions(-)
delete mode 100644 python/mypy.ini
We are specifying >= pylint 2.8.x for several reasons:
1. For setup.cfg support, added in pylint 2.5.x
2. To specify a version that has incompatibly dropped
bad-whitespace checks (2.6.x)
3. 2.7.x fixes "unsubscriptable" warnings in Python 3.9
4. 2.8.x adds a new, incompatible
Python infrastructure as it exists today is not capable reliably of
single-sourcing a package version from a parent directory. The authors
of pip are working to correct this, but as of today this is not possible.
The problem is that when using pip to build and install a python
package, it copies
Add short readmes to python/, python/qemu/, python/qemu/machine,
python/qemu/qmp, and python/qemu/utils that explain the directory
hierarchy. These readmes are visible when browsing the source on
e.g. gitlab/github and are designed to help new developers/users quickly
make sense of the source
We manage cleaning up this resource ourselves. Pylint should shush.
Signed-off-by: John Snow
Reviewed-by: Cleber Rosa
Message-id: 20210517184808.3562549-5-js...@redhat.com
Signed-off-by: John Snow
---
python/qemu/console_socket.py | 1 +
1 file changed, 1 insertion(+)
diff --git
Pylint 2.5.x - 2.7.x have regressions that make import checking
inconsistent, see:
https://github.com/PyCQA/pylint/issues/3609
https://github.com/PyCQA/pylint/issues/3624
https://github.com/PyCQA/pylint/issues/3651
Pinning to 2.4.4 is worse, because it mandates versions of shared
dependencies
Add setup.cfg and setup.py, necessary for installing a package via
pip. Add a ReST document (PACKAGE.rst) explaining the basics of what
this package is for and who to contact for more information. This
document will be used as the landing page for the package on PyPI.
List the subpackages we
When creating a source or binary distribution via 'python3 setup.py
', the VERSION and PACKAGE.rst files aren't bundled by
default. Create a MANIFEST.in file that instructs the build tools to
include these so that installation from these files won't fail.
This is required by 'tox', as well as by
use run() instead of Popen() -- to assert to pylint that we are not
forgetting to close a long-running program.
Signed-off-by: John Snow
Reviewed-by: Cleber Rosa
Tested-by: Cleber Rosa
Message-id: 20210517184808.3562549-4-js...@redhat.com
Signed-off-by: John Snow
---
python/qemu/machine.py |
mypy is kind of weird about how it handles imports. For legacy reasons,
it won't load PEP 420 namespaces, because of logic implemented prior to
that becoming a standard.
So, if you plan on using any, you have to pass
--namespace-packages. Alright, fine.
Signed-off-by: John Snow
Reviewed-by:
Shift the open() call later so that the pylint pragma applies *only* to
that one open() call. Add a note that suggests why this is safe: the
resource is unconditionally cleaned up in _post_shutdown().
_post_shutdown is called after failed launches (see launch()), and
unconditionally after every
We handle this resource rather meticulously in
shutdown/kill/wait/__exit__ et al, through the laborious mechanisms in
_do_shutdown().
Quiet this pylint warning here.
Signed-off-by: John Snow
Reviewed-by: Cleber Rosa
Message-id: 20210517184808.3562549-7-js...@redhat.com
Signed-off-by: John Snow
Fixes pylint warnings.
Signed-off-by: John Snow
Reviewed-by: Cleber Rosa
Reviewed-by: Philippe Mathieu-Daudé
Message-id: 20210517184808.3562549-2-js...@redhat.com
Signed-off-by: John Snow
---
python/qemu/console_socket.py | 10 +-
1 file changed, 5 insertions(+), 5 deletions(-)
diff
Based-on: https://gitlab.com/cleber.gnu/qemu/-/commits/python-next
CI: https://gitlab.com/jsnow/qemu/-/pipelines/309506648
GitLab: https://gitlab.com/jsnow/qemu/-/tree/python-package-mk4
MR: https://gitlab.com/jsnow/qemu/-/merge_requests/7
ABOUT
=
This series factors the python/qemu
One less file resource to manage, and it helps quiet some pylint >=
2.8.0 warnings about not using a with-context manager for the open call.
Signed-off-by: John Snow
Reviewed-by: Philippe Mathieu-Daudé
Reviewed-by: Cleber Rosa
Message-id: 20210517184808.3562549-3-js...@redhat.com
On 5/25/21 4:42 PM, Cleber Rosa wrote:
On Tue, May 25, 2021 at 04:10:55PM -0400, John Snow wrote:
On 5/25/21 3:36 PM, Cleber Rosa wrote:
On Wed, May 12, 2021 at 07:12:39PM -0400, John Snow wrote:
Ignore *Python* build and package output (build, dist, qemu.egg-info);
these files are not
On Tue, May 25, 2021 at 04:10:55PM -0400, John Snow wrote:
> On 5/25/21 3:36 PM, Cleber Rosa wrote:
> > On Wed, May 12, 2021 at 07:12:39PM -0400, John Snow wrote:
> > > Ignore *Python* build and package output (build, dist, qemu.egg-info);
> > > these files are not created as part of a QEMU build.
On Tue, May 25, 2021 at 04:25:37PM -0400, John Snow wrote:
> On 5/25/21 4:15 PM, Cleber Rosa wrote:
> > On Wed, May 12, 2021 at 07:12:41PM -0400, John Snow wrote:
> > > This is intended to be a manually run, non-CI script.
> > >
> > > Use tox to test the linters against all python versions from
On Tue, May 25, 2021 at 03:45:26PM -0400, John Snow wrote:
> On 5/25/21 3:24 PM, Cleber Rosa wrote:
> > On Wed, May 12, 2021 at 07:12:38PM -0400, John Snow wrote:
> > > Add "make venv" to create the pipenv-managed virtual environment that
> > > contains our explicitly pinned dependencies.
> > >
>
On Tue, May 25, 2021 at 01:43:42PM -0400, John Snow wrote:
> On 5/25/21 12:13 PM, Cleber Rosa wrote:
> > On Wed, May 12, 2021 at 07:12:36PM -0400, John Snow wrote:
> > > setuptools doesn't have a formal understanding of development requires,
> > > but it has an optional feataures section. Fine;
On Tue, May 25, 2021 at 01:21:25PM -0400, John Snow wrote:
> On 5/25/21 11:56 AM, Cleber Rosa wrote:
> > On Wed, May 12, 2021 at 07:12:34PM -0400, John Snow wrote:
> > > isort 5.0.0 through 5.0.4 has a bug that causes it to misinterpret
> > > certain "from ..." clauses that are not related to
On 5/25/21 3:55 PM, Cleber Rosa wrote:
On Wed, May 12, 2021 at 07:12:40PM -0400, John Snow wrote:
Add python3.6 to the fedora container image: we need it to run the
linters against that explicit version to make sure we don't break our
minimum version promise.
Add pipenv so that we can fetch
On Tue, May 25, 2021 at 01:14:50PM -0400, John Snow wrote:
> On 5/24/21 10:33 PM, Cleber Rosa wrote:
> > On Wed, May 12, 2021 at 07:12:22PM -0400, John Snow wrote:
> > > Add short readmes to python/, python/qemu/, python/qemu/machine,
> > > python/qemu/qmp, and python/qemu/utils that explain the
On 5/25/21 4:15 PM, Cleber Rosa wrote:
On Wed, May 12, 2021 at 07:12:41PM -0400, John Snow wrote:
This is intended to be a manually run, non-CI script.
Use tox to test the linters against all python versions from 3.6 to
3.9. This will only work if you actually have those versions installed
On Wed, May 12, 2021 at 07:12:41PM -0400, John Snow wrote:
> This is intended to be a manually run, non-CI script.
>
> Use tox to test the linters against all python versions from 3.6 to
> 3.9. This will only work if you actually have those versions installed
> locally, but Fedora makes this
On 5/25/21 3:36 PM, Cleber Rosa wrote:
On Wed, May 12, 2021 at 07:12:39PM -0400, John Snow wrote:
Ignore *Python* build and package output (build, dist, qemu.egg-info);
these files are not created as part of a QEMU build.
Ignore miscellaneous cached python confetti (__pycache__, *.pyc,
On Wed, May 12, 2021 at 07:12:40PM -0400, John Snow wrote:
> Add python3.6 to the fedora container image: we need it to run the
> linters against that explicit version to make sure we don't break our
> minimum version promise.
>
> Add pipenv so that we can fetch precise versions of pip packages
On 5/25/21 3:24 PM, Cleber Rosa wrote:
On Wed, May 12, 2021 at 07:12:38PM -0400, John Snow wrote:
Add "make venv" to create the pipenv-managed virtual environment that
contains our explicitly pinned dependencies.
Add "make check" to run the python linters [in the host execution
environment].
On Wed, May 12, 2021 at 07:12:39PM -0400, John Snow wrote:
> Ignore *Python* build and package output (build, dist, qemu.egg-info);
> these files are not created as part of a QEMU build.
>
> Ignore miscellaneous cached python confetti (__pycache__, *.pyc,
> .mypy_cache).
>
> Ignore .idea
On Wed, May 12, 2021 at 07:12:38PM -0400, John Snow wrote:
> Add "make venv" to create the pipenv-managed virtual environment that
> contains our explicitly pinned dependencies.
>
> Add "make check" to run the python linters [in the host execution
> environment].
>
> Add "make venv-check" which
On Wed, May 12, 2021 at 07:12:37PM -0400, John Snow wrote:
> Try using avocado to manage our various tests; even though right now
> they're only invoking shell scripts and not really running any
> python-native code.
>
> Create tests/, and add shell scripts which call out to mypy, flake8,
>
On 5/25/21 12:13 PM, Cleber Rosa wrote:
On Wed, May 12, 2021 at 07:12:36PM -0400, John Snow wrote:
setuptools doesn't have a formal understanding of development requires,
but it has an optional feataures section. Fine; add a "devel" feature
and add the requirements to it.
To avoid duplication,
* Kevin Wolf (kw...@redhat.com) wrote:
> Commit f61fe11aa6f broke hmp_loadvm() by adding an incorrect negation
> when converting from 0/-errno return values to a bool value. The result
> is that loadvm resumes the VM now if it failed and keeps it stopped if
> it failed. Fix it to restore the old
On 5/25/21 11:50 AM, Cleber Rosa wrote:
On Wed, May 12, 2021 at 07:12:29PM -0400, John Snow wrote:
Instruct flake8 to avoid certain well-known directories created by
python tooling that it ought not check.
Note that at-present, nothing actually creates a ".venv" directory; but
it is in such
On 5/24/21 10:33 PM, Cleber Rosa wrote:
On Wed, May 12, 2021 at 07:12:22PM -0400, John Snow wrote:
Add short readmes to python/, python/qemu/, python/qemu/machine,
python/qemu/qmp, and python/qemu/utils that explain the directory
hierarchy. These readmes are visible when browsing the source on
On 5/25/21 11:56 AM, Cleber Rosa wrote:
On Wed, May 12, 2021 at 07:12:34PM -0400, John Snow wrote:
isort 5.0.0 through 5.0.4 has a bug that causes it to misinterpret
certain "from ..." clauses that are not related to imports.
isort < 5.1.1 has a bug where it does not handle comments near
On Wed, May 12, 2021 at 07:12:31PM -0400, John Snow wrote:
> mypy supports reading its configuration values from a central project
> configuration file; do so.
>
> Signed-off-by: John Snow
> ---
> python/mypy.ini | 4
> python/setup.cfg | 5 +
> 2 files changed, 5 insertions(+), 4
On Wed, May 12, 2021 at 07:12:29PM -0400, John Snow wrote:
> Instruct flake8 to avoid certain well-known directories created by
> python tooling that it ought not check.
>
> Note that at-present, nothing actually creates a ".venv" directory; but
> it is in such widespread usage as a de-facto
Am 24.05.2021 um 18:06 hat Programmingkid geschrieben:
> >>> qemu-system-ppc -usb -device usb-storage,drive=fat16 -drive
> >>> file=fat:rw:fat-type=16:" >>> folder>",id=fat16,format=raw,if=none
> >>>
> On a related topic would you know if it is possible to use fat32
> instead of fat16 for host
On Wed, May 12, 2021 at 07:12:36PM -0400, John Snow wrote:
> setuptools doesn't have a formal understanding of development requires,
> but it has an optional feataures section. Fine; add a "devel" feature
> and add the requirements to it.
>
> To avoid duplication, we can modify pipenv to install
On Wed, May 12, 2021 at 07:12:34PM -0400, John Snow wrote:
> isort 5.0.0 through 5.0.4 has a bug that causes it to misinterpret
> certain "from ..." clauses that are not related to imports.
>
> isort < 5.1.1 has a bug where it does not handle comments near import
> statements correctly.
>
>
On 5/24/21 10:42 PM, Cleber Rosa wrote:
On Wed, May 12, 2021 at 07:12:23PM -0400, John Snow wrote:
When creating a source distribution via 'python3 setup.py sdist', the
VERSION and PACKAGE.rst files aren't bundled by default. Create a
MANIFEST.in file that instructs the build tools to include
> On May 25, 2021, at 2:05 AM, Vladimir Sementsov-Ogievskiy
> wrote:
>
> 24.05.2021 20:33, Programmingkid wrote:
>>> On May 24, 2021, at 12:56 PM, Vladimir Sementsov-Ogievskiy
>>> wrote:
>>>
>>> 24.05.2021 19:06, Programmingkid wrote:
> On May 24, 2021, at 11:55 AM, Vladimir
25.05.2021 13:18, Emanuele Giuseppe Esposito wrote:
On 21/05/2021 19:10, Vladimir Sementsov-Ogievskiy wrote:
18.05.2021 13:07, Emanuele Giuseppe Esposito wrote:
With tasks and calls lock protecting all State fields,
.method is the last BlockCopyState field left unprotected.
Set it as atomic.
On 21/05/2021 17:01, Paolo Bonzini wrote:
On 20/05/21 17:30, Vladimir Sementsov-Ogievskiy wrote:
18.05.2021 13:07, Emanuele Giuseppe Esposito wrote:
As for BlockCopyTask, add a lock to protect BlockCopyCallState
ret and sleep_state fields. Also move ret, finished and cancelled
in the OUT
25.05.2021 13:07, Emanuele Giuseppe Esposito wrote:
On 20/05/2021 17:19, Vladimir Sementsov-Ogievskiy wrote:
18.05.2021 13:07, Emanuele Giuseppe Esposito wrote:
Because the list of tasks is only modified by coroutine
functions, add a CoMutex in order to protect them.
Use the same mutex to
On 21/05/2021 19:10, Vladimir Sementsov-Ogievskiy wrote:
18.05.2021 13:07, Emanuele Giuseppe Esposito wrote:
With tasks and calls lock protecting all State fields,
.method is the last BlockCopyState field left unprotected.
Set it as atomic.
Signed-off-by: Emanuele Giuseppe Esposito
OK,
On 20/05/2021 17:19, Vladimir Sementsov-Ogievskiy wrote:
18.05.2021 13:07, Emanuele Giuseppe Esposito wrote:
Because the list of tasks is only modified by coroutine
functions, add a CoMutex in order to protect them.
Use the same mutex to protect also BlockCopyState in_flight_bytes
field to
25.05.2021 11:50, Max Reitz wrote:
On 19.05.21 08:11, Vladimir Sementsov-Ogievskiy wrote:
18.05.2021 19:39, Max Reitz wrote:
[...]
On 17.05.21 14:07, Vladimir Sementsov-Ogievskiy wrote:
[...]
Not also, that there is another benefit of such thing: we'll implement this
callback in qcow2
On 19.05.21 08:11, Vladimir Sementsov-Ogievskiy wrote:
18.05.2021 19:39, Max Reitz wrote:
[...]
On 17.05.21 14:07, Vladimir Sementsov-Ogievskiy wrote:
[...]
Not also, that there is another benefit of such thing: we'll
implement this callback in qcow2 driver, so that backup will read
24.05.2021 20:33, Programmingkid wrote:
On May 24, 2021, at 12:56 PM, Vladimir Sementsov-Ogievskiy
wrote:
24.05.2021 19:06, Programmingkid wrote:
On May 24, 2021, at 11:55 AM, Vladimir Sementsov-Ogievskiy
wrote:
24.05.2021 18:41, Programmingkid wrote:
On May 24, 2021, at 6:12 AM,
68 matches
Mail list logo
