On Tue, Dec 9, 2025, 12:00 PM Daniel P. Berrangé <[email protected]>
wrote:
> On Fri, Dec 05, 2025 at 01:00:42AM -0500, John Snow wrote:
> > Hello!
> >
> > This series drops the in-tree version of our python-qemu-qmp package
> > ("qemu.qmp") in favor of the version hosted on PyPI, whose repository is
> > located at https://gitlab.com/qemu-project/python-qemu-qmp.
> >
> > GitLab CI: https://gitlab.com/jsnow/qemu/-/pipelines/2197613036
> > (FreeBSD isn't my fault...!)
> >
> > The major effects of this patch series are:
> >
> > 1. qemu.qmp will be installed from PyPI or vendored packages instead of
> > being utilized directly from the qemu.git tree.
>
> This is not getting installed in enough scenarios IMHO.
>
It's hard to trigger an install when you don't use the build system,
though... If you bypass make/meson/ninja entirely I'm not really sure what
I can do to bootstrap the test deps.
> My current workflow is commonly
>
> $ ./configure --target-list=x86_64-softmmu
> $ make
> $ ./scripts/qmp/qmp-shell-wrap
> /var/home/berrange/src/virt/qemu/build
> Traceback (most recent call last):
> File "/var/home/berrange/src/virt/qemu/scripts/qmp/qmp-shell-wrap",
> line 7, in <module>
> from qemu.qmp import qmp_shell
> ModuleNotFoundError: No module named 'qemu.qmp'
>
> Even if I add in a call to 'source build/pyvenv/bin/activate'
> after 'make', I'm still missing the qemu.qmp python module.
>
> AFAICT, qemu.qmp only gets installed in the venv if you run
> 'make check', and IMHO that should not be expected for the
> above usage scenario.
>
"make check-venv", to do just the minimum setup. You don't have to run
check.
We could always add a flag to configure to front-load the testing
environment and scripts, but I thought it was best to avoid adding new
dependencies by default.
Like --with-python-tests or something? Then you'd always have it like you
do with meson and sphinx (with --enable-docs)
> Likewise I can no longer run any of the test programs directly
> without first having run 'make check'. eg what I would currently
> do is:
>
> $ ./configure --target-list=x86_64-softmmu
> $ make
> $ cd build/tests/qemu-iotests
> $ ./check 300
>
> I don't generally run 'make check' as frequently as I
> 'configure && make' as it adds alot of time which is
> not needed usually.
>
> In general I find it tedious having to remember to run
> 'source build/pyvenv/bin/activate' in every terminal
> window where I might want to run a QEMU script that relies
> on python. It is also fairly undesirable too, as I use the
> same terminal window for various purposes, that activating
> the QEMU venv can interfere with non-QEMU python usage.
>
You can run one-offs with build/pyvenv/bin/python3 without activating the
environment,
but you're right that this does require an environment setup step (make
check-venv)
> I would find it helpful if we provided a "run" script in the
> root dir of the build directory that contained the following
>
> $ cat build/run
> #!/bin/sh
>
> set -e
>
> HERE=$(realpath $(dirname $0))
> source $HERE/pyvenv/bin/activate
> exec "$@"
>
> Which would be used as a wrapper to execute scripts with the
> right environment. That would let us keep the simpler workflow,
> and avoid polluting the global terminal environment with the
> qemu venv.
>
> eg to be used as
>
> $ ./configure --target-list=x86_64-softmmu
> $ make
> $ cd build/tests/qemu-iotests
> $ ../../run ./check 300
>
> or
>
> $ ./configure --target-list=x86_64-softmmu
> $ make
> $ ./build/run ./scripts/qmp/qmp-shell-wrap
Shouldn't be too hard to add, honestly. Maybe adding in a "meson compile
pyvenv/pyvenv_tests_group" line at the start to trigger the dep
installation if it hasn't occurred already?
Or maybe just directly invoking mkvenv ensuregroup to avoid ninja
re-configuring things while you develop.
Something like that.
>
>
>
> > 2. There are no new python dependencies checked or installed during
> > configure. All test dependencies are installed post-hoc on an
> > as-needed basis.
> > 3. "make check-venv" is no longer required to be run manually before any
> > test that is integrated with meson; this includes "make check" and
> > "make check-functional".
> > 4. "make check-venv" no longer installs functional test dependencies: it
> > installs only the core suite of python test dependencies.
> > 5. "make check-venv" is now required as a pre-requisite for running
> > device-crash-test and manually executed iotests.
> > 6. Unfortunately, python3-wheel and python3-setuptools are now required
> > on the host system if tests are to be executed and >= Python 3.13 is
> > used.
> > 7. An awful lot of deleted lines of code, and a lot fewer headaches
> > managing two nearly-identical copies of this source code. O:-)
> >
> > Patches 1-5 are build system focused; they set up new pythondeps.toml,
> > mkvenv, and meson systems in preparation for relying on an external
> > qemu.qmp library, but does not yet make the switch.
> >
> > Patches 6-9 are testing and CI focused; they add necessary preparation
> > steps to keep tests running happily once the in-tree qemu.qmp library is
> > removed.
> >
> > Patches 10-15 are build system focused again; they implement the actual
> > switchover to the external qemu.qmp library.
>
> With regards,
> Daniel
> --
> |: https://berrange.com -o-
> https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o-
> https://fstop138.berrange.com :|
> |: https://entangle-photo.org -o-
> https://www.instagram.com/dberrange :|
>
>
