On Thu, Jul 24, 2025 at 3:47 PM Thomas Huth <th...@redhat.com> wrote: > > On 21/07/2025 22.38, John Snow wrote: > > On Thu, Jul 17, 2025 at 4:44 AM Alex Bennée <alex.ben...@linaro.org> wrote: > ... > >> Am I holding it wrong? > >> > >> ➜ ./pyvenv/bin/python ../../tests/functional/test_aarch64_virt.py > >> --help > >> Traceback (most recent call last): > >> File > >> "/home/alex/lsrc/qemu.git/builds/all/../../tests/functional/test_aarch64_virt.py", > >> line 16, in <module> > >> from qemu_test import QemuSystemTest, Asset, > >> exec_command_and_wait_for_pattern > >> File > >> "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/__init__.py", line > >> 14, in <module> > >> from .testcase import QemuBaseTest, QemuUserTest, QemuSystemTest > >> File > >> "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line > >> 26, in <module> > >> from qemu.machine import QEMUMachine > >> ModuleNotFoundError: No module named 'qemu' > >> > >> I thought the point of the venv is we had all the modules we need > >> automatically available to the PYTHONPATH? > > > > As Thomas points out, "qemu" is special since it's already in the > > tree. There has been some dragging-of-feet by yours-truly because > > installing the "qemu" module by default when running configure > > introduces a considerable startup lag time, and the module is not > > actually needed for the simple configuration and building of QEMU - > > only testing. > > > > It's something I want to fix, but must admit to being a bit stumped as > > to how I will bridge that gap long term. Currently, all of the modules > > we need are in the tree with no dependencies, so it can be fixed with > > a simple PYTHONPATH hack. However, if I actually remove the QMP > > library from the tree like I have wanted to, then we need pip to do a > > real install and process dependencies, and this creates some > > complications and extra startup lag. > > Wouldn't it be possible to add the module as a wheel in python/wheels/ ? > That's maybe the easiest solution, isn't it?
The qemu.qmp wheel, yes, because it's purepython. The part I am worried about is that by removing qemu.qmp from the tree, you cannot just use the PYTHONPATH hack anymore to use the remaining in-tree goodies, namely the machine module used extensively throughout testing, because they will now rely on an external dependency. Normally, you'd just install the in-tree stuff like any other package and pip would take care of the qemu.qmp dependency. As handled through mkvenv, it would search our vendored wheels directory in the process and use that wheel, it's not a big deal. The tricky part in my head is that this "pip install" of the in-tree goodies takes a few seconds because pip is quite slow - it technically has to "build" the package before it installs it, so it adds some lag time. Since qemu.qmp isn't actually needed to configure or build QEMU, I thought it'd be inappropriate to add it to those dependencies - and inappropriate to enforce a 3-4 second hit on every configure call whether or not you intend to run tests afterwards. I'm sure it's solvable, I just tried once a while back and hit a wall, got distracted, and haven't revisited it yet... Maybe a good rubber-ducking session would help. > > > Naively, I think adding a "just in time installation of testing > > dependencies" when you go to run a testing command from "make XXXX" > > might be sufficient for us, possibly in conjunction with a configure > > flag that lets you pre-load testing dependencies. > > We could likely re-use "make check-venv" for the functional tests ... it's > already installed in that case. However, you then still have to remember to > call it first before you can run a test directly, without the Makefile > wrappers. > > Thomas >
