Organizing a codebase that has been developed for over 5 years
'organically' is not easy.
With the repository split work, it has become evident that having to
create 2 PRs just to add a new libvirt test is awkward at best (one
tp-libvirt PR for the test itself, another one to add stuff to
virttest.virsh and/or virttest.libvirt_xml, which nearly universally
happens). The same also happens to qemu tests, of course.
It is imperative to make efforts to split, clean and streamline the
available virtualization test APIs. Keeping things streamlined is quite
the challenge, as the sheer amount of code being pushed every day to
virt-test makes difficult to keep paying attention to where all these
functions are being thrown to. When you have 50 open PRs open, you are
not exactly picky with regards to where people decide to put code.
That's how I'm approaching the problem:
(1) - Go over the current virttest namespace, and assessing which
libraries can be transferred over existing test providers
(2) - Create the PRs to make transfer/fix references
(3) - Somehow come up with 'rules' about what belongs to test providers
-virttest -autotest.
(4) - Have API police ready to enforce the rules.
(1) and (2) are far easier than (3) and (4). We did try to have rules
before, but the project grew, the rules were written only to docstrings
of modules, and the pressure on delivering time reviews with few
reviewers made any sort of rule enforcement to fail.
This means you are welcome to help with creating rules for API placement
(3), and mechanisms to enforce such rules (4). Meanwhile, I'll carry on
with (1) and (2).
Ok, so going with (1), here's the list of APIs and my assessment of each
of them:
.
|-- aexpect.py -core
|-- arch.py -core
|-- asset.py -core
|-- base_installer.py -core
|-- bootstrap.py -core
|-- build_helper.py -core
|-- cartesian_config.py -core
|-- data_dir.py -core
|-- element_path.py -drop candidate with autotest 0.16 RPM
|-- element_tree.py -drop candidate with autotest 0.16 RPM
|-- env_process.py -core
|-- funcatexit.py -core
|-- gluster.py -core (?)
|-- guest_agent.py -tp-qemu
|-- http_server.py -core
|-- installer.py -core
|-- iscsi.py -core
|-- libvirt_storage.py -tp-libvirt
|-- libvirt_vm.py -core
|-- libvirt_xml -tp-libvirt
|-- lvm.py -tp-qemu
|-- lvsb_base.py -tp-libvirt
|-- lvsb.py -tp-libvirt
|-- lvsbs.py -tp-libvirt
|-- nfs.py -core
|-- openvswitch.py -tp-qemu
|-- ovirt.py -tp-libvirt
|-- ovs_utils.py -tp-libvirt
|-- passfd.c -core
|-- passfd.py -core
|-- passfd_setup.py -core
|-- postprocess_iozone.py -make a core autotest lib, move this
to staging
|-- ppm_utils.py -core (?)
|-- propcan.py -core
|-- qemu_devices -core (qemu_vm depends on it)
|-- qemu_installer.py -core (installer depends on it)
|-- qemu_io.py -tp-qemu
|-- qemu_monitor.py -core (qemu_vm depends on it)
|-- qemu_qtree.py -tp-qemu
|-- qemu_storage.py -core (interestingly, tp-libvirt
depends on it)
|-- qemu_virtio_port.py -core (qemu_vm depends on it)
|-- qemu_vm.py -core
|-- remote.py -core
|-- RFBDes.py -future tp-spice (this badly needs a
rename)
|-- rss_client.py -core
|-- scheduler.py -get rid of this (very likely bitrot)
|-- staging -we want to get rid of this the 1st
chance we have
|-- standalone_test.py -core
|-- step_editor.py -tp-qemu
|-- storage.py -core
|-- syslog_server.py -core (unattended_install depends on it)
|-- tests -core
|-- test_setup.py -core (this library should be torn
apart and APIs go to other libs)
|-- utils_conn.py -tp-libvirt (although one could make
a point for core)
|-- utils_disk.py -core
|-- utils_env.py -core
|-- utils_libguestfs.py -tp-libvirt
|-- utils_libvirtd.py -tp-libvirt
|-- utils_misc.py -core
|-- utils_netperf.py -tp-qemu (although one could make a
point for core)
|-- utils_net.py -core
|-- utils_params.py -core
|-- utils_selinux.py -core
|-- utils_spice.py -future tp-spice
|-- utils_test -This should be split into each provider
| |-- __init__.py -core (stuff should be moved out here)
| |-- libguestfs.py -tp-libvirt
| |-- libvirt.py -tp-libvirt
| `-- qemu.py -tp-qemu
|-- utils_v2v.py -tp-libvirt
|-- versionable_class.py -tp-qemu
|-- version.py -core
|-- video_maker.py -core
|-- virsh.py -tp-libvirt
|-- virt_vm.py -core
|-- xml_utils.py -core (move to staging)
`-- yumrepo.py -core (installer depends on it)
Which gives me the first 2 big API moves:
What goes to tp-qemu
|-- guest_agent.py
|-- lvm.py
|-- openvswitch.py
|-- qemu_io.py
|-- qemu_qtree.py
|-- step_editor.py
|-- utils_netperf.py
|-- utils_test
| `-- qemu.py
|-- versionable_class.py
What goes to tp-libvirt
|-- libvirt_storage.py
|-- libvirt_xml
|-- lvsb_base.py
|-- lvsb.py
|-- lvsbs.py
|-- ovirt.py
|-- ovs_utils.py
|-- utils_conn.py
|-- utils_libguestfs.py
|-- utils_libvirtd.py
|-- utils_test
| |-- libguestfs.py
| |-- libvirt.py
|-- utils_v2v.py
|-- virsh.py
That is a good first pass, and it may require some tweaking. Let me know
if you disagree with something, and we can get it fixed. They are going
to the respective 'provider_lib' namespace (although calling the library
simply 'provider' is appealing).
There's a number of things that are possibly worth tackling, such as
getting rid of the many utils_* prefixes on library names. I am the one
that came up with utils_* in the first place, but in my defense, I had
no better idea at the time.
Thanks, and let me know if you have any problems/and/or suggestions.
_______________________________________________
Virt-test-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/virt-test-devel
