Re: [openstack-dev] [NFV][CI][third-party] The plan to bring up Snabb NFV CI for Juno-3
- Original Message - From: Luke Gorrie l...@snabb.co To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Greetings fellow NFV'stas! I would like to explain and solicit feedback on our plan to support a new open source NFV system in Juno. This work is approved as low-priority/best-effort for Juno-3. (Yes, we do understand that we are fighting the odds in terms of the Juno schedule.) We are developing a practical open source NFV implementation for OpenStack. This is for people who want to run tens of millions of packets per second through Virtio-net on each compute node. The work involves contributing code upstream to a dependent chain of projects: snabbswitch - QEMU - Libvirt - Nova - Neutron Hi Luke, I've added the [third-party] tag as well to ensure this catches the broadest segment of relevant people. Probably a stupid question from me - are any modifications to upstream Open vSwitch required to support Snabb? Thanks, Steve Recently we had a breakthrough: QEMU upstream merged the vhost-user feature that we developed and this convinced the kind maintainers of Libvirt, Nova, and Neutron to let us target code to them in parallel. Now Libvirt has accepted our code upstream too and the last pieces are Nova and Neutron. (Then we can start work on Version 2.) Previously our upstreaming effort has been obstructed: people understandably wanted to see our QEMU code accepted before they would take us seriously. So it is an exciting time for us and our upstreaming work. Just now we have ramped up our OpenStack development effort in response to getting approved for Juno-3. Michele Paolino has joined in: he is experienced with Libvirt and is the one who upstreamed our code there. Nikolay Nikolaev is joining in too: he did the bulk of the development on vhost-user and the upstreaming of it into QEMU. Here is what the three of us are working on for Juno-3: * VIF_VHOSTUSER support in Nova. https://blueprints.launchpad.net/nova/+spec/vif-vhostuser * Snabb NFV mech driver for Neutron. https://blueprints.launchpad.net/neutron/+spec/snabb-nfv-mech-driver * NFV CI: OpenStack 3rd party CI that covers our entire software ecosystem (snabbswitch + QEMU + Libvirt + Nova + Neutron). We are already getting great support from the community. Thank you everybody for that, and meta-thankyou to the people who setup the NFV subgroup which has been a fantastic enabler. For the code changes, the ball is in our court now to get them into shape in time. For the CI, I think it's worth having a discussion to make sure we are on the same page and have the same expectations. Have you already attempted to solicit some core reviewers in Nova and Neutron with an Here is how I visualize our ideal NFV CI for Juno: * Run Tempest tests for Nova and Neutron. * Test with the relevant versions of Libvirt, QEMU, and snabbswitch. * Test with NFV-oriented features that are upstream in OpenStack. * Test with NFV-oriented changes that are not yet upstream e.g. Neutron QoS API. * Operate reliably with a strong track record. * Be easy for other people to replicate if they want to run their own NFV CI. This CI should then provide assurance for us that our whole ecosystem is running compatibly, for OpenStack that the code going upstream is continuously tested, and for end users that the software they plan to deploy works (either based on our tests, if they are deploying the same software that we use, or based on their own tests if they want to operate a customised CI). How does this CI idea sound to the community and to others who are interested in related NFV-oriented features? That was quite a brain-dump... we have been working on this for quite some time but mostly on the parts outside of the OpenStack tree until now. For more information about our open source NFV project you can read the humble home page: http://snabb.co/nfv.html and if you want to talk nuts and bolts you can find us on Github: https://github.com/SnabbCo/snabbswitch and Google Groups: https://groups.google.com/forum/#!forum/snabb-devel We are independent open source developers and we are working to support Deutsche Telekom's TeraStream NFV project. Cheers! -Luke ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Steve Gordon, RHCE Sr. Technical Product Manager, Red Hat Enterprise Linux OpenStack Platform ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [NFV][CI][third-party] The plan to bring up Snabb NFV CI for Juno-3
Hi Steve, On 29 July 2014 17:21, Steve Gordon sgor...@redhat.com wrote: I've added the [third-party] tag as well to ensure this catches the broadest segment of relevant people. Thanks! are any modifications to upstream Open vSwitch required to support Snabb? Good question. No, this uses a separate vswitch called Snabb Switch. Snabb Switch is a small user-space program that you assign some network interfaces to. It runs independent of any other networking you are doing on other ports (OVS, DPDK-OVS, SR-IOV, etc). Have you already attempted to solicit some core reviewers in Nova and Neutron How does one normally do that? We are getting help but I am not exactly sure how people have found us beyond chat in #openstack-nfv :-). Two Neutron core reviewers are making the requirements there very clear to us, both on the code and the CI. One Nova core reviewer is helping us too. I would like to better understand CI requirements on the Nova side (e.g. does the Neutron tempest testing regime provide adequate coverage for Nova or do we need to do more?). This is our first contribution to Nova so there is a risk that we overlook something important. Cheers, -Luke ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [NFV][CI][third-party] The plan to bring up Snabb NFV CI for Juno-3
- Original Message - From: Luke Gorrie l...@snabb.co To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Cc: Nikolay Nikolaev n.nikol...@virtualopensystems.com Sent: Tuesday, July 29, 2014 12:28:55 PM Subject: Re: [openstack-dev] [NFV][CI][third-party] The plan to bring up Snabb NFV CI for Juno-3 Hi Steve, On 29 July 2014 17:21, Steve Gordon sgor...@redhat.com wrote: I've added the [third-party] tag as well to ensure this catches the broadest segment of relevant people. Thanks! are any modifications to upstream Open vSwitch required to support Snabb? Good question. No, this uses a separate vswitch called Snabb Switch. Snabb Switch is a small user-space program that you assign some network interfaces to. It runs independent of any other networking you are doing on other ports (OVS, DPDK-OVS, SR-IOV, etc). OK, that's what I thought - thanks for confirming. Have you already attempted to solicit some core reviewers in Nova and Neutron How does one normally do that? We are getting help but I am not exactly sure how people have found us beyond chat in #openstack-nfv :-). Two Neutron core reviewers are making the requirements there very clear to us, both on the code and the CI. Ideally this is fairly organic, the NFV group certainly serves as a forum for coordinating what work is out there in this space but ultimately it is necessary to also liaise with and meet the expectations of the actual projects (e.g. Neutron and Nova in this case). This is an area where we as a group still have a lot of work to do, in my opinion. I see that there is some discussion to this end in the code submission for the new mechanism driver: https://review.openstack.org/#/c/95711/ It appears to me the expectation/desire from Mark and Maru here is to see a lot more justification of the use cases for this driver and the direction of the current implementation so while it is positive that it got the attention of some core reviewers early there are some hard questions that will need to be answered to continue making progress with this for Juno. One Nova core reviewer is helping us too. I would like to better understand CI requirements on the Nova side (e.g. does the Neutron tempest testing regime provide adequate coverage for Nova or do we need to do more?). This is our first contribution to Nova so there is a risk that we overlook something important. Typically third party CI is only provided/required for Nova when adding/maintaining a new hypervisor driver - at least that seems to be the case so far. I know in your earlier email you mentioned also wanting to use this third party CI to also test a number of other scenarios, particularly: * Test with NFV-oriented features that are upstream in OpenStack. * Test with NFV-oriented changes that are not yet upstream e.g. Neutron QoS API. I am not sure how this would work - perhaps I misunderstand what you are proposing? As it stands the third-party CI jobs ideally run on each change *submitted* to gerrit so features that are not yet *merged* still receive this CI testing today both from the CI managed by infra and the existing third-party CI jobs? Or are you simply highlighting that you wish to test same with snabbswitch? Just not quite understanding why these were called out as separate cases. Thanks, Steve ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
