On Fri, May 13, 2022 at 09:35:23AM +0100, Daniel P. Berrangé wrote: > On Thu, May 12, 2022 at 08:06:00PM -0400, John Snow wrote: > > RFC: This is a very early, crude attempt at switching over to an > > external Python package dependency for QMP. This series does not > > actually make the switch in and of itself, but instead just switches to > > the paradigm of using a venv in general to install the QEMU python > > packages instead of using PYTHONPATH to load them from the source tree. > > > > (By installing the package, we can process dependencies.) > > > > I'm sending it to the list so I can show you some of what's ugly so far > > and my notes on how I might make it less ugly. > > > > (1) This doesn't trigger venv creation *from* iotests, it merely prints > > a friendly error message if "make check-venv" has not been run > > first. Not the greatest. > > So if we run the sequence > > mkdir build > cd build > ../configure > make > ./tests/qemu-iotests/check 001 > > It won't work anymore, until we 'make check-venv' (or simply > 'make check') ? > > I'm somewhat inclined to say that venv should be created > unconditionally by default. ie a plain 'make' should always > everything needed to be able to invoke the tests directly. > > > (2) This isn't acceptable for SRPM builds, because it uses PyPI to fetch > > packages just-in-time. My thought is to use an environment variable like > > QEMU_CHECK_NO_INTERNET that changes the behavior of the venv setup > > process. We can use "--system-site-packages" as an argument to venv > > creation and "--no-index" as an argument to pip installation to achieve > > good behavior in SRPM building scenarios. It'd be up to the spec-writer > > to opt into that behavior. > > I think I'd expect --system-site-packages to be the default behaviour. > We expect QEMU to be compatible with the packages available in the > distros that we're targetting. So if the dev has the python packages > installed from their distro, we should be using them preferentially. > > This is similar to how we bundle slirp/capstone/etc, but will > preferentially use the distro version if it is available.
AFAICT from testing it, when '--system-site-packages' is set for the venv, then 'pip install' appears to end up being a no-op if the package is already present in the host, but installs it if missing. IOW, if we default to --system-site-packages, but still also run 'pip install', we should "do the right thing". It'll use any distro packages that are available, and augment with stuff from pip. In the no-op case, pip will still try to consult the pipy servers to fetch version info IIUC, so we need to be able to stop that. So I'd suggest a --python-env arg taking three values * "auto" (the default) - add --system-site-packages, but also run 'pip install'. Good for developers day-to-day * "system" - add --system-site-packages but never run 'pip install'. Good for formal package builds. * "pip" - don't add --system-site-packages, always run 'pip install'. Good for testing compatibility with bleeding edge python versions, or if explicit full independance from host python install is desired. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
