Re: [openstack-dev] [TripleO][CI] Isolated Network Testing
On 10/12/2016 05:58 PM, Dan Sneddon wrote: I recently evaluated our needs for testing coverage for TripleO isolated networking. I wanted to post my thoughts on the matter for discussion, which will hopefully lead to a shared understanding of what improvements we need to make. I think we can cover the majority of end-user requirements by testing the following minimum scenarios: 1. single-nic-vlans (one nic, all VLANs trunked, great for virt and POCs) 2. Provisioning + bond (to test basic bonding functionality) 3. Bonded provisioning (perhaps one bond with all VLANs) 4. Spine and leaf (in the near future) Within those four scenarios, we should ensure that we are testing both IPv4 and IPv6, and both traditional Neutron SNAT/Floating IPs and DVR. The first scenario is well covered. I think scenario 2 is covered by a review posted by Ben Nemec recently [1]. I would very much like to see us testing scenario 3 with a resilient bond for the provisioning interface as well. This used to require LACP and fallback to a single link, but I believe recent changes to the PXE boot images may allow this over links without special switch configuration. I'm currently doing testing in my lab, I hope I can work with the TripleO CI team to help make this happen upstream. Providing two provisioning interfaces to the overcloud nodes is pretty trivial now, so that shouldn't be a problem. Spine and leaf (routed networks) support may require specific configuration of the routing hardware in order to support PXE booting across router boundaries. Specifically, a DHCP proxy needs to be configured in order to forward DHCP requests from a remote VLAN to the Undercloud. If this is not possible in our bare-metal CI environments, then we may need to develop a method of testing this in OVB. I'm very interested in finding out about whether it may be possible to have DHCP proxy (or "DHCP helper-address") configured on the router hardware for CI VLANs. If we can deploy this in bare metal, I think it will save us a lot of time and effort over recreating a routed environment in OVB. I believe we could use use Open Daylight or another OpenFlow controller to simulate routers in virtual environments, or perhaps use dnsmasq in DHCP proxy mode on the OVB host to forward requests from the various bridges representing remote VLANs to the Undercloud br-ctlplane bridge. But it would be a fair amount of work to put that together. Note that we can't do VLANs in OVB right now. Our version of Neutron doesn't support VLAN-aware instances. I know I've seen discussion of adding that to Neutron so it may be something we can do some day, but I have no idea when it will be possible. I don't believe we currently test IPv6 or DVR (please correct me if I'm mistaken). Do we have plans in the works to add these to any jobs? The ipv6 jobs are unfortunately experimental right now, but we're discussing how to get it back into the regular jobs because it's clearly something we need (we fixed ipv6 and then broke it again in less than a month because we didn't have a voting job). It seems like we could add DVR to pretty much any of the existing jobs. I guess it might require different net-iso templates for the compute nodes? They're already wired up to the external network though so I think the environment should support it. Finally, I wonder if we need to test any exotic configurations, such as OVS+DPDK, OpenDaylight, etc. OVS+DPDK would require compatible hardware. I'm interested in hearing feedback about how critical this would be in the grand scheme of things. It isn't yet clear to me that OVS+DPDK is going to have widespread adoption, but I do recognize that there are some NFV users that depend on this technology. Things that require hardware are pretty much a no-go for upstream CI, other than maybe a periodic job. The hardware requirements to run even a basic nonha job on baremetal for every patch would be absurd. Plus, giving access to baremetal is problematic from a security perspective too. I think this would have to be a third-party CI job that could be triggered by some trusted group of people on specific patches. OpenDaylight does not require hardware changes AFAIK, but the drivers and network interface config differs significantly from ML2+OVS. I'm helping some ODL developers make changes that will allow deployment via TripleO, but these changes won't be tested by CI. Of course, there are elements of OVS+DPDK and ODL that get tested as part of Neutron CI, but now we are implementing TripleO-based deployment of these technologies, I wonder if we should endeavor to test them in CI. I suppose that begs the question, if we are testing those, then why not Contrail, etc.? I don't know where we draw the line, but it seems that we might want to at least periodically test deploying some other Neutron drivers via TripleO. For stuff that is virt-friendly, I think we could probably add some ovb scenario jobs like
Re: [openstack-dev] [TripleO][CI] Isolated Network Testing
On Thu, Oct 13, 2016 at 4:28 AM, Dan Sneddon wrote: > I recently evaluated our needs for testing coverage for TripleO > isolated networking. I wanted to post my thoughts on the matter for > discussion, which will hopefully lead to a shared understanding of what > improvements we need to make. I think we can cover the majority of > end-user requirements by testing the following minimum scenarios: > > 1. single-nic-vlans (one nic, all VLANs trunked, great for virt and POCs) > > 2. Provisioning + bond (to test basic bonding functionality) > > 3. Bonded provisioning (perhaps one bond with all VLANs) > > 4. Spine and leaf (in the near future) > > Is linux bridges, in the list? Within those four scenarios, we should ensure that we are testing both > IPv4 and IPv6, and both traditional Neutron SNAT/Floating IPs and DVR. > > The first scenario is well covered. I think scenario 2 is covered by a > review posted by Ben Nemec recently [1]. > > I would very much like to see us testing scenario 3 with a resilient > bond for the provisioning interface as well. This used to require LACP > and fallback to a single link, but I believe recent changes to the PXE > boot images may allow this over links without special switch > configuration. I'm currently doing testing in my lab, I hope I can work > with the TripleO CI team to help make this happen upstream. > > Spine and leaf (routed networks) support may require specific > configuration of the routing hardware in order to support PXE booting > across router boundaries. Specifically, a DHCP proxy needs to be > configured in order to forward DHCP requests from a remote VLAN to the > Undercloud. If this is not possible in our bare-metal CI environments, > then we may need to develop a method of testing this in OVB. > > I'm very interested in finding out about whether it may be possible to > have DHCP proxy (or "DHCP helper-address") configured on the router > hardware for CI VLANs. If we can deploy this in bare metal, I think it > will save us a lot of time and effort over recreating a routed > environment in OVB. I believe we could use use Open Daylight or another > OpenFlow controller to simulate routers in virtual environments, or > perhaps use dnsmasq in DHCP proxy mode on the OVB host to forward > requests from the various bridges representing remote VLANs to the > Undercloud br-ctlplane bridge. But it would be a fair amount of work to > put that together. > > I don't believe we currently test IPv6 or DVR (please correct me if I'm > mistaken). Do we have plans in the works to add these to any jobs? > > Finally, I wonder if we need to test any exotic configurations, such as > OVS+DPDK, OpenDaylight, etc. > > We have SR-IOV as well, which requires specific nics. Could Tripleo be a good candidate to have different sets of 3rd party CI, wherein we can run the specific test cases. > OVS+DPDK would require compatible hardware. I'm interested in hearing > feedback about how critical this would be in the grand scheme of > things. It isn't yet clear to me that OVS+DPDK is going to have > widespread adoption, but I do recognize that there are some NFV users > that depend on this technology. > > OpenDaylight does not require hardware changes AFAIK, but the drivers > and network interface config differs significantly from ML2+OVS. I'm > helping some ODL developers make changes that will allow deployment via > TripleO, but these changes won't be tested by CI. > > Of course, there are elements of OVS+DPDK and ODL that get tested as > part of Neutron CI, but now we are implementing TripleO-based > deployment of these technologies, I wonder if we should endeavor to > test them in CI. I suppose that begs the question, if we are testing > those, then why not Contrail, etc.? I don't know where we draw the > line, but it seems that we might want to at least periodically test > deploying some other Neutron drivers via TripleO. > > Adding OVN to the list as well. regards /sanjay > [1] - https://review.openstack.org/#/c/385562 > > -- > Dan Sneddon | Senior Principal OpenStack Engineer > dsned...@redhat.com | redhat.com/openstack > dsneddon:irc| @dxs:twitter > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [TripleO][CI] Isolated Network Testing
I recently evaluated our needs for testing coverage for TripleO isolated networking. I wanted to post my thoughts on the matter for discussion, which will hopefully lead to a shared understanding of what improvements we need to make. I think we can cover the majority of end-user requirements by testing the following minimum scenarios: 1. single-nic-vlans (one nic, all VLANs trunked, great for virt and POCs) 2. Provisioning + bond (to test basic bonding functionality) 3. Bonded provisioning (perhaps one bond with all VLANs) 4. Spine and leaf (in the near future) Within those four scenarios, we should ensure that we are testing both IPv4 and IPv6, and both traditional Neutron SNAT/Floating IPs and DVR. The first scenario is well covered. I think scenario 2 is covered by a review posted by Ben Nemec recently [1]. I would very much like to see us testing scenario 3 with a resilient bond for the provisioning interface as well. This used to require LACP and fallback to a single link, but I believe recent changes to the PXE boot images may allow this over links without special switch configuration. I'm currently doing testing in my lab, I hope I can work with the TripleO CI team to help make this happen upstream. Spine and leaf (routed networks) support may require specific configuration of the routing hardware in order to support PXE booting across router boundaries. Specifically, a DHCP proxy needs to be configured in order to forward DHCP requests from a remote VLAN to the Undercloud. If this is not possible in our bare-metal CI environments, then we may need to develop a method of testing this in OVB. I'm very interested in finding out about whether it may be possible to have DHCP proxy (or "DHCP helper-address") configured on the router hardware for CI VLANs. If we can deploy this in bare metal, I think it will save us a lot of time and effort over recreating a routed environment in OVB. I believe we could use use Open Daylight or another OpenFlow controller to simulate routers in virtual environments, or perhaps use dnsmasq in DHCP proxy mode on the OVB host to forward requests from the various bridges representing remote VLANs to the Undercloud br-ctlplane bridge. But it would be a fair amount of work to put that together. I don't believe we currently test IPv6 or DVR (please correct me if I'm mistaken). Do we have plans in the works to add these to any jobs? Finally, I wonder if we need to test any exotic configurations, such as OVS+DPDK, OpenDaylight, etc. OVS+DPDK would require compatible hardware. I'm interested in hearing feedback about how critical this would be in the grand scheme of things. It isn't yet clear to me that OVS+DPDK is going to have widespread adoption, but I do recognize that there are some NFV users that depend on this technology. OpenDaylight does not require hardware changes AFAIK, but the drivers and network interface config differs significantly from ML2+OVS. I'm helping some ODL developers make changes that will allow deployment via TripleO, but these changes won't be tested by CI. Of course, there are elements of OVS+DPDK and ODL that get tested as part of Neutron CI, but now we are implementing TripleO-based deployment of these technologies, I wonder if we should endeavor to test them in CI. I suppose that begs the question, if we are testing those, then why not Contrail, etc.? I don't know where we draw the line, but it seems that we might want to at least periodically test deploying some other Neutron drivers via TripleO. [1] - https://review.openstack.org/#/c/385562 -- Dan Sneddon | Senior Principal OpenStack Engineer dsned...@redhat.com | redhat.com/openstack dsneddon:irc| @dxs:twitter __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev