On Tue, Jun 28, 2022 at 1:58 PM Thomas Huth <th...@redhat.com> wrote: > > On 28/06/2022 10.23, Daniel P. Berrangé wrote: > > On Tue, Jun 28, 2022 at 01:21:35PM +0530, Ani Sinha wrote: > >> On Tue, Jun 28, 2022 at 1:19 PM Daniel P. Berrangé <berra...@redhat.com> > >> wrote: > >>> > >>> On Tue, Jun 28, 2022 at 09:25:35AM +0200, Thomas Huth wrote: > >>>> On 28/06/2022 09.10, Michael S. Tsirkin wrote: > >>>>> On Tue, Jun 28, 2022 at 09:03:33AM +0200, Thomas Huth wrote: > >>>>>>>>>>>> No problem with that. So that's venv. But do we need pip and > >>>>>>>>>>>> pulling > >>>>>>>>>>>> packages from the net during testing? > >>>>>>>>>>> > >>>>>>>>>>> We do that too. See requirements.txt in tests/ > >>>>>>>>>>> Following two are downloaded: > >>>>>>>>>>> avocado-framework==88.1 > >>>>>>>>>>> pycdlib==1.11.0 > >>>>>>>>>>> > >>>>>>>>>>> Also see this line in Makefie.include: > >>>>>>>>>>> > >>>>>>>>>>> $(call quiet-venv-pip,install -r $(TESTS_VENV_REQ)) > >>>>>>>>>> > >>>>>>>>>> Right but that's avocado since it pulls lots of stuff from > >>>>>>>>>> the net anyway. > >>>>>>>>>> Are the libraries in question not packaged on major distros? > >>>>>>>>> > >>>>>>>>> Currently I only need this: > >>>>>>>>> https://github.com/python-tap/tappy > >>>>>>>>> which is the basic TAP processing library for python. > >>>>>>>>> > >>>>>>>>> It seems its only installed through pip: > >>>>>>>>> https://tappy.readthedocs.io/en/latest/ > >>>>>>>>> > >>>>>>>>> I do not think this is packaged by default. It's such a basic > >>>>>>>>> library > >>>>>>>>> for parsing test output that maybe we can keep this somewhere within > >>>>>>>>> the python src tree? Not sure ... > >>>>>>>> > >>>>>>>> It's pretty small for sure. Another submodule? > >>>>>>> > >>>>>>> Unlike BITS, this one is likely going to be maintained for a while and > >>>>>>> will receive new releases through > >>>>>>> https://pypi.org/project/tap.py/ > >>>>>>> so forking is OK but someone has to keep this updated. > >>>>>>> > >>>>>>> I am open to anything. Whatever feels right is fine to me. > >>>>>> > >>>>>> John Snow is currently working on the "Pythonification" of various QEMU > >>>>>> bits, I think you should loop him into this discussion, too. > >>>>>> > >>>>>> Thomas > >>>>> > >>>>> submodule does not mean we fork necessarily. We could have > >>>>> all options: check for the module and use it if there, if not > >>>>> use one from system if not there install with pip .. > >>>>> But yea, I'm not sure what's best either. > >>>> > >>>> submodules create a dependency on an internet connection, too. So before > >>>> you > >>>> add yet another submodule (which have a couple of other disadvantages), I > >>>> think you could also directly use the venv here. > >>> > >>> Definitely not submodules. > >>> > >>> We need to get out of the mindset that submodules are needed for every new > >>> dependancy we add. Submodules are only appropriate if the external project > >>> is designed to be used as a copylib (eg the keycodemapdb tool), or if we > >>> need to bundle in order to prevent a regression for previously deployed > >>> QEMU installs where the dependancy is known not to exist on all our > >>> supported platforms. > >>> > >>> This does not apply in this case, because the proposed use of tappy is > >>> merely for a test case. Meson just needs to check if tappy exists and if > >>> it does, then use it, otherwise skip the tests that need it. The user can > >>> arrange to install tappy, as they do with the majority of other deps. > >>> > >>> If John's venv stuff is relevant, then we don't even need the meson > >>> checks, > >>> just delegate to the venv setup. > >>> > >>> Regardless, no submodules are needed or desirable. > >> > >> What about keeping biosbits stuff? Source or pre-built. > > > > Shipping them as pre-built binaries in QEMU is not a viable option > > IMHO, especially for grub as a GPL'd project we need to be extremely > > clear about the exact corresponding source and build process for any > > binary. > > > > For this kind of thing I would generally expect the distro to provide > > packages that we consume. Looking at biosbits I see it is itself > > bundling a bunch more 3rd party projects, libffi, grub2, and including > > even an ancient version of python as a submodule. > > > > So bundling a pre-built biosbits in QEMU appears to mean that we're in > > turn going to unexpectedly bundle a bunch of other 3rd party projects > > too, all with dubious license compliance. I don't think this looks like > > something we should have in qemu.git or qemu tarballs. It will also > > make it challenging for the distro to take biosbits at all, unless > > those 3rd party bundles can be eliminated in favour of using existing > > builds their have packaged for grub, python, libffi, etc. > > So if this depends on some third party binary bits, I think this is pretty > similar to the tests in the avocado directory ... there we download third > party binaries, too... Wouldn't it make sense to adapt your tests to that > framework?
I do not want to bring in the whole avocado framework because it would unnecessarily make things complicated. I just need the qemu machine python library and that is enough. For downloading third party stuff, we can simply wget things from somewhere. > > Thomas >