Reviewed: https://review.opendev.org/c/openstack/nova/+/862769 Committed: https://opendev.org/openstack/nova/commit/ecb11043e9c2a798950d63b55cd08684b1d77c4a Submitter: "Zuul (22348)" Branch: master
commit ecb11043e9c2a798950d63b55cd08684b1d77c4a Author: melanie witt <[email protected]> Date: Thu Oct 27 01:43:55 2022 +0000 Add mock to avoid loading guestfs in unit test We recently discovered that when the perfect conditions are present where: * libguestfs-dev/el and guestfs python bindings are installed and * unit tests are not being run in a venv or guestfs python bindings are installed in the tox venv the test will end up loading the guestfs module and try to call the real guestfs and possibly libvirt and fail because of it. Our unit tests shouldn't be loading modules like guestfs, so this adds proper mocking to the test along with a poison fixture that will prevent future accidental imports of such modules. Closes-Bug: #1994913 Change-Id: I676ee1fd33cf053681a07448759c28f0f2ad79d1 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1994913 Title: Unit tests can fail if python guestfs is installed in the test environment Status in OpenStack Compute (nova): Fix Released Bug description: Ran into this downstream [1] where a unit test failed because it actually tried to call libvirt, which of course failed: libguestfs: error: could not create appliance through libvirt. Try running qemu directly without libvirt using this environment variable: export LIBGUESTFS_BACKEND=direct Original error from libvirt: internal error: qemu unexpectedly closed the monitor: 2022-10-22T13:20:42.059709Z qemu-kvm: -blockdev {"node-name":"libvirt-2-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"qcow2","file":"libvirt-2-storage","backing":null}: Image is not in qcow2 format [code=1 int1=-1] {5} nova.tests.unit.virt.disk.test_api.APITestCase.test_can_resize_need_fs_type_specified [191.116248s] ... FAILED Captured traceback: ~~~~~~~~~~~~~~~~~~~ Traceback (most recent call last): File "/home/zuul/src/code.engineering.redhat.com/nova/nova/tests/unit/virt/disk/test_api.py", line 47, in test_can_resize_need_fs_type_specified self.assertFalse(api.is_image_extendable(image)) File "/home/zuul/src/code.engineering.redhat.com/nova/nova/virt/disk/api.py", line 190, in is_image_extendable fs.setup(mount=False) File "/home/zuul/src/code.engineering.redhat.com/nova/nova/virt/disk/vfs/guestfs.py", line 235, in setup self.handle.launch() File "/usr/lib/python3.9/site-packages/eventlet/tpool.py", line 190, in doit result = proxy_call(self._autowrap, f, *args, **kwargs) File "/usr/lib/python3.9/site-packages/eventlet/tpool.py", line 148, in proxy_call rv = execute(f, *args, **kwargs) File "/usr/lib/python3.9/site-packages/eventlet/tpool.py", line 121, in execute rv = e.wait() File "/usr/lib/python3.9/site-packages/eventlet/event.py", line 125, in wait result = hub.switch() File "/usr/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 313, in switch return self.greenlet.switch() File "/usr/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 365, in run self.wait(sleep_time) File "/usr/lib/python3.9/site-packages/eventlet/hubs/poll.py", line 80, in wait presult = self.do_poll(seconds) File "/usr/lib/python3.9/site-packages/eventlet/hubs/epolls.py", line 31, in do_poll return self.poll.poll(seconds) File "/usr/lib/python3.9/site-packages/fixtures/_fixtures/timeout.py", line 52, in signal_handler raise TimeoutException() fixtures._fixtures.timeout.TimeoutException That ^ was with libguestfs 1.46.1. Repro steps (not identical traceback but close): 1. Make sure you have libguestfs header files $ sudo apt install libguestfs-dev 2. Enter the unit test venv $ source .tox/py38/bin/activate 3. Install the libguestfs python bindings into the venv. This has to be done by tarball [2]. Note that the bindings were not built for python 3 until version 1.40 [3] $ pip install http://libguestfs.org/download/python/guestfs-1.40.2.tar.gz 4. Run the unit test $ tox -epy38 test_can_resize_need_fs_type_specified 5. The test should fail The code in nova will call libvirt if guestfs is installed [4] and if it is not installed, it will emit a warning and move on [5]. In my repro environment it didn't get as far as a call to libvirt though. Upstream the test doesn't fail because neither libvirt nor guestfs are installed in the CI environment for unit test jobs. A unit test shouldn't be able to call out to real guestfs or libvirt and it wasn't mocked out properly. We can address it by adding mocking to the unit test and create a fixture that will poison future imports by unit or func tests. [1] https://bugzilla.redhat.com/show_bug.cgi?id=2135856 [2] https://libguestfs.org/guestfs-python.3.html#using-python-bindings-in-a-virtualenv [3] https://bugzilla.redhat.com/show_bug.cgi?id=1627964#c15 [4] https://github.com/openstack/nova/blob/b1958b7cfa6b8aca5b76b3f133627bb733d29f00/nova/virt/disk/vfs/guestfs.py#L66-L70 [5] https://github.com/openstack/nova/blob/b1958b7cfa6b8aca5b76b3f133627bb733d29f00/nova/virt/disk/api.py#L189-L201 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1994913/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
