On Tue, Jul 24, 2018, at 10:21 AM, Artom Lifshitz wrote: > On Tue, Jul 24, 2018 at 12:30 PM, Clark Boylan <cboy...@sapwetik.org> wrote: > > On Tue, Jul 24, 2018, at 9:23 AM, Artom Lifshitz wrote: > >> Hey all, > >> > >> tl;dr Humbly requesting a handful of nodes to run NFV tests in CI > >> > >> Intel has their NFV tests tempest plugin [1] and manages a third party > >> CI for Nova. Two of the cores on that project (Stephen Finucane and > >> Sean Mooney) have now moved to Red Hat, but the point still stands > >> that there's a need and a use case for testing things like NUMA > >> topologies, CPU pinning and hugepages. > >> > >> At Red Hat, we also have a similar tempest plugin project [2] that we > >> use for downstream whitebox testing. The scope is a bit bigger than > >> just NFV, but the main use case is still testing NFV code in an > >> automated way. > >> > >> Given that there's a clear need for this sort of whitebox testing, I > >> would like to humbly request a handful of nodes (in the 3 to 5 range) > >> from infra to run an "official" Nova NFV CI. The code doing the > >> testing would initially be the current Intel plugin, bug we could have > >> a separate discussion about keeping "Intel" in the name or forking > >> and/or renaming it to something more vendor-neutral. > > > > The way you request nodes from Infra is through your Zuul configuration. > > Add jobs to a project to run tests on the node labels that you want. > > Aha, thanks, I'll look into that. I was coming from a place of > complete ignorance about infra. > > > > I'm guessing this process doesn't work for NFV tests because you have > > specific hardware requirements that are not met by our current VM resources? > > If that is the case it would probably be best to start by documenting what > > is required and where the existing VM resources fall > > short. > > Well, it should be possible to do most of what we'd like with nested > virt and virtual NUMA topologies, though things like hugepages will > need host configuration, specifically the kernel boot command [1]. Is > that possible with the nodes we have?
https://docs.openstack.org/infra/manual/testing.html attempts to give you an idea for what is currently available via the test environments. Nested virt has historically been painful because not all clouds support it and those that do did not do so in a reliable way (VMs and possibly hypervisors would crash). This has gotten better recently as nested virt is something more people have an interest in getting working but it is still hit and miss particularly as you use newer kernels in guests. I think if we can continue to work together with our clouds (thank you limestone, OVH, and vexxhost!) we may be able to work out nested virt that is redundant across multiple clouds. We will likely need individuals willing to keep caring for that though and debug problems when the next release of your favorite distro shows up. Can you get by with qemu or is nested virt required? As for hugepages, I've done a quick survey of cpuinfo across our clouds and all seem to have pse available but not all have pdpe1gb available. Are you using 1GB hugepages? Keep in mind that the test VMs only have 8GB of memory total. As for booting with special kernel parameters you can have your job make those modifications to the test environment then reboot the test environment within the job. There is some Zuul specific housekeeping that needs to be done post reboot, we can figure that out if we decide to go down this route. Would your setup work with 2M hugepages? > > > In general though we operate on top of donated cloud resources, and if > > those do not work we will have to identify a source of resources that would > > work. > > Right, as always it comes down to resources and money. I believe > historically Red Hat has been opposed to running an upstream third > party CI (this is by no means an official Red Hat position, just > remembering what I think I heard), but I can always see what I can do. > > [1] > https://docs.openstack.org/nova/latest/admin/huge-pages.html#enabling-huge-pages-on-the-host __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
