On Thu, May 5, 2022, 4:51 AM Kevin Wolf <[email protected]> wrote: > Am 04.05.2022 um 21:38 hat John Snow geschrieben: > > Howdy! > > > > So, I want to finally delete python/qemu/qmp from qemu.git, and this > > creates a small problem -- namely, iotests needs access to it in order > > to run the python-based tests. > > > > What I think needs to happen is that we create a virtual environment > > that installs python/qemu/. The reason this cannot be done with > > PYTHONPATH alone anymore is because the qmp package itself won't be > > there anymore, we need an installer like `pip` to actually fetch it > > for us and put it somewhere. (i.e., we need to process the > > dependencies of python/qemu now and can't treat it as a pre-installed > > location.) > > > > Avocado tests are already creating a venv for the purposes of > > installing and running Avocado. We can amend e.g. "../../python" to > > tests/requirements.txt and the Avocado environment is A-OK good-to-go. > > The Makefile magic for avocado tests creates a venv-per-build. > > "per build" sounded pretty bad, because I thought it meant building a > new venv every time I run either 'make' or the tests. This would > obviously be very noticable for short-running tests like some iotests. > But fortunately this is not what it does, it keeps the venv around in > the build directory. So it's only per build directory really, which I > guess is fine. >
Whoops, yeah. I meant per build directory. In contrast to the Python unit tests themselves, which create some test venvs tied directly to the source directory and are build-agnostic. > > It seems to work well enough. One thing to note here is that the > > supported invocation for avocado tests is only through the Makefile, > > which handles creating and entering the venv to make the command > > seamless. > > > > iotests, however, manages its own execution environment with > > testenv.py, and we support running iotests from outside of the > > Makefile, for example by going to $build/tests/qemu-iotests and > > running ./check. > > Yes, and I agree that this is important. > Figured as much. Not plotting to take this away, I promise. Just getting my requirements straight before I spend time concocting something. > > Now ... I could update testenv.py to be smart enough to create and > > enter a python venv, and have even prototyped this. It seems to work > > pretty well! This approach seemed like the least invasive to how > > iotests are expected to be run and used. But a downside with this > > approach is that now avocado tests and iotests are each managing their > > own python venv. Worse, vm-tests and device-crash-test are still > > unhandled entirely. > > Is there a reason why testenv.py couldn't enter just the shared venv in > build/tests/venv? > It can, but it'd have to be made first so it can enter it. I figured this would only happen "on-demand", like how check-avocado works, so I'd need some way for iotests and the Makefile to share the venv creation code, which is certainly an option. > If not, I guess it would be enough if iotests just checks that the venv > exists and all of the dependencies are there in the right version and > error out if not, telling the user to run 'make check-venv'. > I will spend unreasonable engineering hours avoiding this, if only for pride. I want everything to be as seamless and easy as it's always been. > Or actually, it could just unconditionally run 'make check-venv' by > itself, which is probably easier to implement than checking the > dependencies and more convenient for the user, too. > Oh, that's one way to get them to share the venv-creation pathway. Hadn't occurred to me, but this might be easy to do. > (One more detail: 'make check-venv GIT_SUBMODULES_ACTION=ignore' seems > to make it pretty much instantaneous if the venv is current, but leaving > the submodule mechanism enabled adds a noticable delay.) > Noted. > > I'd like to find a solution where I create a unified python testing > > venv tied to the build shared by avocado, iotests, vm-tests and > > device-crash-test. I'm not completely sure how exactly I'll manage > > that right now, but I wanted to throw this out there in case there are > > some requirements I might be overlooking. > > > > I think vm-tests and avocado-tests can both have a venv created for > > them and activated before the test runs. device-crash-test I believe > > will need a script change in the gitlab ci yaml. iotests is somewhat > > unique in that it needs to run both by manual invocation and from > > makefile invocations. If I want a shared VM between all of these, I'll > > need to isolate the create-and-enter-venv logic somewhere where it can > > be shared both inside and outside of a Makefile. > > If just calling 'make' isn't an option, then moving that part out into a > separate script probably wouldn't be too hard either. But 'make' has the > advantage that it already contains the check if the venv is already > there and requirements.txt hasn't changed, which you would have to > replicate otherwise. > Yeah, exactly. > Kevin > Thanks for the ideas! As of now, I've got a branch where I've moved vm-tests over onto the same venv with avocado-tests. If I can bridge the gap to iotests, I'll be pretty happy.
