Re: [openstack-dev] [nova] NUMA + SR-IOV
> -Original Message- > From: Nikola Đipanov [mailto:ndipa...@redhat.com] > Sent: Thursday, March 24, 2016 4:34 PM > To: Sergey Nikitin ; OpenStack Development Mailing > List (not for usage questions) > Cc: Czesnowicz, Przemyslaw > Subject: Re: [openstack-dev] [nova] NUMA + SR-IOV > > On 03/24/2016 04:18 PM, Sergey Nikitin wrote: > > > > Hi, folks. > > > > I want to start a discussion about NUMA + SR-IOV environment. I have a > > two-sockets server. It has two NUMA nodes and only one SR-IOV PCI > > device. This device is associated with the first NUMA node. I booted a > > set of VMs with SR-IOV support. Each of these VMs was booted on the > > first NUMA node. As I understand it happened for better performance > > (VM should be booted in NUMA node which has PCI device for this VM) > [1]. > > > > But this behavior leaves my 2-sockets machines half-populated. What if > > I don't care about SR-IOV performance? I just want every VM from *any* > > of NUMA nodes to use this single SR-IOV PCI device. > > > > But I can't do it because of behavior of numa_topology_filter. In this > > filter we want to know if current host has required PCI device [2]. > > But we want to have this device *only* in some numa cell on this host. > > It is hardcoded here [3]. If we do *not* pass variable "cells" to the > > method > > support_requests() [4] we will boot VM on the current host, if it has > > required PCI device *on host* (maybe not in the same NUMA node). > > > > So my question is: > > Is it correct that we *always* want to boot VM in NUMA node associated > > with requested PCI device and user has no choice? > > Or should we give a choice to the user and let him boot a VM with PCI > > device, associated with another NUMA node? > > The rationale for choosing this behavior was that if you are requiring a NUMA topology for your VM and you request an SRIOV device as well then this is an high performance application and it should be configured appropriately. Similarly if you request hugepages your VM will be confined to one NUMA (unless specified otherwise) node and if there is no single NUMA node with enough resources it won't be created. > > This has come up before, and the fact that it keeps coming up tells me that > we should probably do something about it. > > Potentially it makes sense to be lax by default unless user specifies that > they > want to make sure that the device is on the same NUMA node, but that is > not backwards compatible. > > It does not make sense to ask user to specify that they don't care IMHO, as > unless you know there is a problem (and users have nowhere near enough > information to tell), there is no reason for you to specify it - it's just not > sensible UI IMHO. > Yes this did come up few times, having a way to specify a requirement is probably a good idea. If it would be done the way you propose that would change the behavior for existing users, not sure how big problem this is. Przemek > My 0.02 cents. > > N. > > > > > [1] > > https://specs.openstack.org/openstack/nova- > specs/specs/kilo/implemente > > d/input-output-based-numa-scheduling.html > > [2] > > > https://github.com/openstack/nova/blob/master/nova/scheduler/filters/n > > uma_topology_filter.py#L85 > > [3] > > > https://github.com/openstack/nova/blob/master/nova/virt/hardware.py#L > 1 > > 246-L1247 [4] > > https://github.com/openstack/nova/blob/master/nova/pci/stats.py#L277 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [networking-ovs-dpdk] VM creation fails with Unexpected vif_type=binding_failed
Hi Praveen, There’s been some changes recently to networking-ovs-dpdk, it no longer host’s a mech driver as the openviswitch mech driver in Neutron supports vhost-user ports. I guess something went wrong and the version of Neutron is not matching networking-ovs-dpdk. Can you post commit ids of Neutron and networking-ovs-dpdk. The other possibility is that the Neutron agent is not running/died on the compute node. Check with: neutron agent-list Przemek From: Praveen MANKARA RADHAKRISHNAN [mailto:praveen.mank...@6wind.com] Sent: Tuesday, November 24, 2015 12:18 PM To: openstack-dev@lists.openstack.org Subject: [openstack-dev] [networking-ovs-dpdk] VM creation fails with Unexpected vif_type=binding_failed Hi, Am trying to set up an open stack (kilo) installation using ovs-dpdk through devstack installation. I have followed the " https://github.com/openstack/networking-ovs-dpdk/blob/master/doc/source/getstarted.rst " documentation. I used the same versions as in documentation (fedora21, with right kernel). My openstack installation is successful in both controller and compute. I have used example local.conf given in the documentation. But if i try to spawn the VM. I am having the following error. "NovaException: Unexpected vif_type=binding_failed" It would be really helpful if you can point out how to debug and fix this error. Thanks Praveen __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [networking-ovs-dpdk]
Hi Samta, This usually means that the vSwitch is not running/has crashed. Can you check in /opt/stack/logs/ovs-vswitchd.log ? There should be an error msg there. Regards Przemek > -Original Message- > From: Samta Rangare [mailto:samtarang...@gmail.com] > Sent: Monday, November 9, 2015 1:51 PM > To: OpenStack Development Mailing List (not for usage questions) > Subject: [openstack-dev] [networking-ovs-dpdk] > > Hello Everyone, > > I am installing devstack with networking-ovs-dpdk. The local.conf exactly > looks like the one is available in /opt/stack/networking-ovs- > dpdk/doc/source/_downloads/local.conf.single_node. > So I believe all the necessary configuration will be taken care. > > However I am stuck at place where devstack is trying to set external-id ($ > sudo ovs-vsctl br-set-external-id br-ex bridge-id br-ex). As soon as it hits > at > this place it's just hangs forever. I tried commenting this line from > lib/neutron_plugin/ml2 (I know this is wrong) and then all services came up > except ovs-dpdk agent and ovs agent. > > BTW I am deploying it in ubuntu 14.04. Any pointer will be really helpful. > > Thanks, > Samta > > __ > > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: OpenStack-dev- > requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][neutron] Create VM using port-create vs nova boot only?
>There seemed to be two ways to create a VM via cli: > >1) use neutron command to create a port first and then use nova command to >attach the vm to that port(neutron port-create.. >followed by nova boot --nic >port-id=) >2)Just use nova command and a port will implicitly be created for you(nova >boot --nic net-id=net-uuid). > >My question is : is #2 sufficient enough to cover all the scenarios? In other >words, if we are not allowed to use #1(can only use >#2 to create vm), would >we miss anything? You wouldn't be able to set the vnic_type of the port. Setting vnic_type=direct is required to boot vm's with SRIOV ports. Regards Przemek __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Feature Freeze Exception Request (libvirt vhostuser vif driver)
Hi, I would like to request FFE for vhostuser vif driver. 2 reviews : https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/libvirt-vif-vhost-user,n,z BP: https://blueprints.launchpad.net/nova/+spec/libvirt-vif-vhost-user Spec: https://review.openstack.org/138736 Blueprint was approved but it's status was changed because of FF. Vhostuser is a Qemu feature that allows fastpath into the VM for userspace vSwitches. The changes are small and mostly contained to libvirt driver. Vhostuser support was proposed for Juno by Snabb switch guys but didn't make it, this implementation supports their usecase as well . Thanks Przemek __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][neutron] Passthrough of PF's from SR-IOV capable devices.
Hi > 1) If the device is a "normal" PCI device, but is a network card, am I still > able to > take advantage of the advanced syntax added circa Juno to define the > relationship between that card and a given physical network so that the > scheduler can place accordingly (and does this still use the ML2 mech drvier > for > SR-IOV even though it's a "normal" device. Actually libvirt won't allow using "normal" PCI devices for network interfaces into VM. Following error is thrown by libvirt 1.2.9.1: libvirtError: unsupported configuration: Interface type hostdev is currently supported on SR-IOV Virtual Functions only I don't know why libvirt prohibits that. But we should prohibit that on Openstack side as well. > > 2) There is no functional reason from a Libvirt/Qemu perspective that I > couldn't > pass through a PF to a guest, and some users have expressed surprise to me > when they have run into this check in the Nova driver. I assume in the initial > implementation this was prevented to avoid a whole heap of fun additional > logic > that is required if this is allowed (e.g. check that no VFs from the PF being > requested are already in use, remove all the associated VFs from the pool when > assigning the PF, who gets allowed to use PFs versus VFs etc.). Am I correct > here > or is there another reason that this would be undesirable to allow in future - > assuming such checks can also be designed - that I am missing? > I think that is correct. But even if the additional logic was implemented it wouldn't work because of how libvirt behaves currently. Regards Przemek > Thanks, > > Steve > > _ > _ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] request spec freeze exception for Api add vnic_type support
Hi, I'd like to request an spec freeze exception for "Api add vnic_type support". https://review.openstack.org/#/c/138808/ There is agreement in SRIOV community that this a very needed change. This change will simplify using sriov (or other vnic_types) and make it more flexible. Thanks, Przemek __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][neutron]VIF_VHOSTUSER
Hi Ian, Spec for “Support vhost-user in libvirt vif driver” [1] has been approved for Kilo. We should have code available by EOW. We are also working on a mechanism driver for Neutron as well [2]. We started working on a 3rd party CI. Regards Przemek [1] https://review.openstack.org/#/c/138736/ [2] https://review.openstack.org/#/c/138742/ From: Ian Wells [mailto:ijw.ubu...@cack.org.uk] Sent: Saturday, January 10, 2015 1:15 AM To: OpenStack Development Mailing List Subject: [openstack-dev] [nova][neutron]VIF_VHOSTUSER Once more, I'd like to revisit the VIF_VHOSTUSER discussion [1]. I still think this is worth getting into Nova's libvirt driver - specifically because there's actually no way to distribute this as an extension; since we removed the plugin mechanism for VIF drivers, it absolutely requires a code change in the libvirt driver. This means that there's no graceful way of distributing an aftermarket VHOSTUSER driver for libvirt. The standing counterargument to adding it is that nothing in the upstream or 3rd party CI would currently test the VIF_VHOSTUSER code. I'm not sure that's a showstopper, given the code is zero risk to anyone when it's not being used, and clearly is going to be experimental when it's enabled. So, Nova cores, would it be possible to incorporate this without a corresponding driver in base Neutron? Cheers, -- Ian. [1] https://review.openstack.org/#/c/96140/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt-driver-numa-placement
> -Original Message- > From: Nikola Đipanov [mailto:ndipa...@redhat.com] > Sent: Thursday, September 04, 2014 4:22 PM > To: openstack-dev@lists.openstack.org > Subject: Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt- > driver-numa-placement > > On 09/04/2014 04:51 PM, Murray, Paul (HP Cloud) wrote: > > > > > > On 4 September 2014 14:07, Nikola Đipanov wrote: > > > > On 09/04/2014 02:31 PM, Sean Dague wrote: > > > >> On 09/04/2014 07:58 AM, Nikola Đipanov wrote: > > > >>> Hi team, > > > >>> > > > >>> I am requesting the exception for the feature from the subject (find > > > >>> specs at [1] and outstanding changes at [2]). > > > >>> > > > >>> Some reasons why we may want to grant it: > > > >>> > > > >>> First of all all patches have been approved in time and just lost > >>> the > > > >>> gate race. > > > >>> > > > >>> Rejecting it makes little sense really, as it has been commented on > >>> by a > > > >>> good chunk of the core team, most of the invasive stuff (db > >>> migrations > > > >>> for example) has already merged, and the few parts that may seem > > > >>> contentious have either been discussed and agreed upon [3], or can > > > >>> easily be addressed in subsequent bug fixes. > > > >>> > > > >>> It would be very beneficial to merge it so that we actually get real > > > >>> testing on the feature ASAP (scheduling features are not tested in > >>> the > > > >>> gate so we need to rely on downstream/3rd party/user testing for those). > > > >> > > > >> This statement bugs me. It seems kind of backwards to say we should > > > >> merge a thing that we don't have a good upstream test plan on and put > >> it > > > >> in a release so that the testing will happen only in the downstream case. > > > >> > > > > > > > > The objective reality is that many other things have not had upstream > > > > testing for a long time (anything that requires more than 1 compute > > node > > > > in Nova for example, and any scheduling feature - as I mention clearly > > > > above), so not sure how that is backwards from any reasonable point. > > > > > > > > Thanks to folks using them, it is still kept working and bugs get fixed. > > > > Getting features into the hands of users is extremely important... > > > > > > > >> Anyway, not enough to -1 it, but enough to at least say something. > > > >> > > > > > > > > .. but I do not want to get into the discussion about software testing > > > > here, not the place really. > > > > > > > > However, I do think it is very harmful to respond to FFE request with > > > > such blanket statements and generalizations, if only for the message > > it > > > > sends to the contributors (that we really care more about upholding > > our > > > > own myths as a community than users and features). > > > > > > > > > > > > I believe you brought this up as one of your justifications for the FFE. > > When I read your statement it does sound as though you want to put > > experimental code in at the final release. I am sure that is not what > > you had in mind, but I am also sure you can also understand Sean's > > point of view. His point is clear and pertinent to your request. > > > > > > > > As the person responsible for Nova in HP I will be interested to see > > how it operates in practice. I can assure you we will do extensive > > testing on it before it goes into the wild and we will not put it into > > practice if we are not happy. > > > > That is awesome and we as a project are lucky to have that! I would not want > things put into practice that users can't use or see huge flaws with. > > I can't help but read this as you being OK with the feature going ahead, > though > :). > > N. We (Intel) are as well very interested in this feature and would like to see it land in Juno. If FFE is granted we will test this feature before release and work with Nikola on fixing bugs if any are found. Regards Przemek > > > > > > > Paul > > > > > > > > Paul Murray > > > > Nova Technical Lead, HP Cloud > > > > +44 117 312 9309 > > > > Hewlett-Packard Limited registered Office: Cain Road, Bracknell, Berks > > RG12 1HN Registered No: 690597 England. The contents of this message > > and any attachments to it are confidential and may be legally > > privileged. If you have received this message in error, you should > > delete it from your system immediately and advise the sender. To any > > recipient of this message within HP, unless otherwise stated you > > should consider this message and attachments as "HP CONFIDENTIAL". > > > > > > > > > > > > ___ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailin
[openstack-dev] [nova][Spec Freeze Exception]Support dpdkvhost in ovs vif bindings
Hi Nova Cores, I would like to ask for spec approval deadline exception for: https://review.openstack.org/#/c/95805/2 This feature allows using DPDK enabled Open vSwitch with Openstack. This is an important feature for NFV workloads that require high performance network I/O. If the spec is approved, implementation should be straight forward and should not disrupt any other work happening in Nova. Thanks, Przemek -- Intel Shannon Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 Business address: Dromore House, East Park, Shannon, Co. Clare This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ML2] Support dpdk ovs with ml2 plugin
I don't think this is a usecase that could be supported right now. There will be multiple issues with running two ovs instances on the node, e.g. how to manage two sets of userspace utilities, two ovsdb servers etc. Also there would be some limitations from how ml2 plugin does port binding (different segmentation ids would have to be used for the two ovs instances) This could be done if ovs was able to run two datapaths at the same time (kernel and dpdk enabled userspace datapath). I would like to concentrate on the more simple usecase where some nodes are optimized for high perf net i/o Thanks Przemek -Original Message- From: Mathieu Rohon [mailto:mathieu.ro...@gmail.com] Sent: Friday, July 11, 2014 10:30 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][ML2] Support dpdk ovs with ml2 plugin A simple usecase could be to have a compute node able start VM with optimized net I/O or standard net I/O, depending on the network flavor ordered for this VM. On Fri, Jul 11, 2014 at 11:16 AM, Czesnowicz, Przemyslaw wrote: > > > Can you explain whats the use case for running both ovs and userspace > ovs on the same host? > > > > Thanks > > Przemek > > From: loy wolfe [mailto:loywo...@gmail.com] > Sent: Friday, July 11, 2014 3:17 AM > > > To: OpenStack Development Mailing List (not for usage questions) > Subject: Re: [openstack-dev] [Neutron][ML2] Support dpdk ovs with ml2 > plugin > > > > +1 > > > > It's totally different between ovs and userspace ovs. > > also, there is strong need to keep ovs even we have a userspace ovs in > the same host > > > > > > -- > > > Intel Shannon Limited > Registered in Ireland > Registered Office: Collinstown Industrial Park, Leixlip, County > Kildare Registered Number: 308263 Business address: Dromore House, > East Park, Shannon, Co. Clare > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Intel Shannon Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 Business address: Dromore House, East Park, Shannon, Co. Clare This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ML2] Support dpdk ovs with ml2 plugin
Hi, We were looking at this solution in the beginning, but it’s won’t work with opendaylight. With opendaylight there is no agent running on the node so this info would have to be provided by opendaylight. Thanks Przemek From: Irena Berezovsky [mailto:ire...@mellanox.com] Sent: Sunday, July 13, 2014 8:31 AM To: Czesnowicz, Przemyslaw; OpenStack Development Mailing List (not for usage questions) Cc: Mooney, Sean K Subject: RE: [openstack-dev] [Neutron][ML2] Support dpdk ovs with ml2 plugin Hi, For agent way to notify server regarding node specific info, you can leverage the periodic state report that neutron agent sends to the neutron Server. As an option, the ML2 Mechanism Driver can check that agent report and depending on the datapath_type, update vif_details. This can be done similar to bridge_mappings: https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers/mech_openvswitch.py#43 BR, Irena From: Czesnowicz, Przemyslaw [mailto:przemyslaw.czesnow...@intel.com] Sent: Thursday, July 10, 2014 6:20 PM To: Irena Berezovsky; OpenStack Development Mailing List (not for usage questions) Cc: Mooney, Sean K Subject: RE: [openstack-dev] [Neutron][ML2] Support dpdk ovs with ml2 plugin Hi, Thanks for Your answers. Yep using binding:vif_details makes more sense. We would like to reuse VIF_TYPE_OVS and modify the nova to use the userspace vhost when ‘use_dpdk’ flag is present. What we are missing is how to inform the ml2 plugin/mechanism drivers when to put that ‘use_dpdk’ flag into vif_details. On the node ovs_neutron_agent could look up datapath_type in ovsdb, but how can we provide that info to the plugin? Currently there is no mechanism to get node specific info into the ml2 plugin (or at least we don’t see any). Any ideas on how this could be implemented? Regards Przemek From: Irena Berezovsky [mailto:ire...@mellanox.com] Sent: Thursday, July 10, 2014 8:08 AM To: OpenStack Development Mailing List (not for usage questions); Czesnowicz, Przemyslaw Cc: Mooney, Sean K Subject: RE: [openstack-dev] [Neutron][ML2] Support dpdk ovs with ml2 plugin Hi, For passing information from neutron to nova VIF Driver, you should use binding:vif_details dictionary. You may not require new VIF_TYPE, but can leverage the existing VIF_TYPE_OVS, and add ‘use_dpdk’ in vif_details dictionary. This will require some rework of the existing libvirt vif_driver VIF_TYPE_OVS. Binding:profile is considered as input dictionary that is used to pass information required for port binding on Server side. You may use binding:profile to pass in a dpdk ovs request, so it will be taken into port binding consideration by ML2 plugin. I am not sure regarding new vnic_type, since it will require port owner to pass in the requested type. Is it your intention? Should the port owner be aware of dpdk ovs usage? There is also VM scheduling consideration that if certain vnic_type is requested, VM should be scheduled on the node that can satisfy the request. Regards, Irena From: loy wolfe [mailto:loywo...@gmail.com] Sent: Thursday, July 10, 2014 6:00 AM To: OpenStack Development Mailing List (not for usage questions) Cc: Mooney, Sean K Subject: Re: [openstack-dev] [Neutron][ML2] Support dpdk ovs with ml2 plugin i think both a new vnic_type and a new vif_type should be added. now vnic has three types: normal, direct, macvtap, then we need a new type of "uservhost". as for vif_type, now we have VIF_TYPE_OVS, VIF_TYPE_QBH/QBG, VIF_HW_VEB, so we need a new VIF_TYPE_USEROVS I don't think it's a good idea to directly reuse ovs agent, for we have to consider use cases that ovs and userovs co-exists. Now it's a little painful to fork and write a new agent, but it will be easier when ML2 agent BP is merged in the future. (https://etherpad.openstack.org/p/modular-l2-agent-outline) On Wed, Jul 9, 2014 at 11:08 PM, Czesnowicz, Przemyslaw mailto:przemyslaw.czesnow...@intel.com>> wrote: Hi We (Intel Openstack team) would like to add support for dpdk based userspace openvswitch using mech_openvswitch and mech_odl from ML2 plugin. The dpdk enabled ovs comes in two flavours one is netdev incorporated into vanilla ovs the other is a fork of ovs with a dpdk datapath (https://github.com/01org/dpdk-ovs ). Both flavours use userspace vhost mechanism to connect the VMs to the switch. Our initial approach was to extend ovs vif bindings in nova and add a config parameter to specify when userspace vhost should be used. Spec : https://review.openstack.org/95805 Code: https://review.openstack.org/100256 Nova devs rejected this approach saying that Neutron should pass all necessary information to nova to select vif bindings. Currently we are looking for a way to pass information from Neutron to Nova that dpdk enabled ovs is being used while still being able to use mech_openvswitch and ovs_neutron_agent or mech_odl. We thought of two possible solutions: 1. Use
Re: [openstack-dev] [Neutron][ML2] Support dpdk ovs with ml2 plugin
Can you explain whats the use case for running both ovs and userspace ovs on the same host? Thanks Przemek From: loy wolfe [mailto:loywo...@gmail.com] Sent: Friday, July 11, 2014 3:17 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][ML2] Support dpdk ovs with ml2 plugin +1 It's totally different between ovs and userspace ovs. also, there is strong need to keep ovs even we have a userspace ovs in the same host -- Intel Shannon Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 Business address: Dromore House, East Park, Shannon, Co. Clare This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ML2] Support dpdk ovs with ml2 plugin
Great, I added this subject to next meetings agenda. Regards Przemek -Original Message- From: Kyle Mestery [mailto:mest...@noironetworks.com] Sent: Thursday, July 10, 2014 2:32 PM To: OpenStack Development Mailing List (not for usage questions) Cc: Mooney, Sean K Subject: Re: [openstack-dev] [Neutron][ML2] Support dpdk ovs with ml2 plugin I'd recommend adding this to the weekly Neutron ML2 meeting agenda [1] and discussing it there. The ML2 sub-team leads (rkukura and Sukhdev) are both at the Neutron mid-cycle meeting this week, so I'd suggest next week's meeting. Thanks! Kyle [1] https://wiki.openstack.org/wiki/Meetings/ML2 On Wed, Jul 9, 2014 at 10:08 AM, Czesnowicz, Przemyslaw wrote: > Hi > > > > We (Intel Openstack team) would like to add support for dpdk based > userspace openvswitch using mech_openvswitch and mech_odl from ML2 plugin. > > The dpdk enabled ovs comes in two flavours one is netdev incorporated > into vanilla ovs the other is a fork of ovs with a dpdk datapath > (https://github.com/01org/dpdk-ovs ). > > Both flavours use userspace vhost mechanism to connect the VMs to the > switch. > > > > Our initial approach was to extend ovs vif bindings in nova and add a > config parameter to specify when userspace vhost should be used. > > Spec : https://review.openstack.org/95805 > > Code: https://review.openstack.org/100256 > > > > Nova devs rejected this approach saying that Neutron should pass all > necessary information to nova to select vif bindings. > > > > Currently we are looking for a way to pass information from Neutron to > Nova that dpdk enabled ovs is being used while still being able to use > mech_openvswitch and ovs_neutron_agent or mech_odl. > > > > We thought of two possible solutions: > > 1. Use binding_profile to provide node specific info to nova. > > Agent rpc api would be extended to allow agents to send node profile > to neutron plugin. > > That info would be stored in db and passed to nova when binding on > this specific host is requested. > > This could be used to support our use case or pass other info to nova > (i.e name of integration bridge) > > > > 2. Let mech_openvswitch and mech_odl detect what binding type to use. > > When asked for port binding mech_openvswitch and mech_odl would call > the agent or odl to check what bindings to use (VIF_TYPE_OVS or > VIF_TYPE_DPDKVHOST) > > > > So, what would be the best way to support our usecase, is it one of > the above ? > > > > Best regards > > Przemek > > -- > Intel Shannon Limited > Registered in Ireland > Registered Office: Collinstown Industrial Park, Leixlip, County > Kildare Registered Number: 308263 Business address: Dromore House, > East Park, Shannon, Co. Clare > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Intel Shannon Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 Business address: Dromore House, East Park, Shannon, Co. Clare This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ML2] Support dpdk ovs with ml2 plugin
Hi, Thanks for Your answers. Yep using binding:vif_details makes more sense. We would like to reuse VIF_TYPE_OVS and modify the nova to use the userspace vhost when ‘use_dpdk’ flag is present. What we are missing is how to inform the ml2 plugin/mechanism drivers when to put that ‘use_dpdk’ flag into vif_details. On the node ovs_neutron_agent could look up datapath_type in ovsdb, but how can we provide that info to the plugin? Currently there is no mechanism to get node specific info into the ml2 plugin (or at least we don’t see any). Any ideas on how this could be implemented? Regards Przemek From: Irena Berezovsky [mailto:ire...@mellanox.com] Sent: Thursday, July 10, 2014 8:08 AM To: OpenStack Development Mailing List (not for usage questions); Czesnowicz, Przemyslaw Cc: Mooney, Sean K Subject: RE: [openstack-dev] [Neutron][ML2] Support dpdk ovs with ml2 plugin Hi, For passing information from neutron to nova VIF Driver, you should use binding:vif_details dictionary. You may not require new VIF_TYPE, but can leverage the existing VIF_TYPE_OVS, and add ‘use_dpdk’ in vif_details dictionary. This will require some rework of the existing libvirt vif_driver VIF_TYPE_OVS. Binding:profile is considered as input dictionary that is used to pass information required for port binding on Server side. You may use binding:profile to pass in a dpdk ovs request, so it will be taken into port binding consideration by ML2 plugin. I am not sure regarding new vnic_type, since it will require port owner to pass in the requested type. Is it your intention? Should the port owner be aware of dpdk ovs usage? There is also VM scheduling consideration that if certain vnic_type is requested, VM should be scheduled on the node that can satisfy the request. Regards, Irena From: loy wolfe [mailto:loywo...@gmail.com] Sent: Thursday, July 10, 2014 6:00 AM To: OpenStack Development Mailing List (not for usage questions) Cc: Mooney, Sean K Subject: Re: [openstack-dev] [Neutron][ML2] Support dpdk ovs with ml2 plugin i think both a new vnic_type and a new vif_type should be added. now vnic has three types: normal, direct, macvtap, then we need a new type of "uservhost". as for vif_type, now we have VIF_TYPE_OVS, VIF_TYPE_QBH/QBG, VIF_HW_VEB, so we need a new VIF_TYPE_USEROVS I don't think it's a good idea to directly reuse ovs agent, for we have to consider use cases that ovs and userovs co-exists. Now it's a little painful to fork and write a new agent, but it will be easier when ML2 agent BP is merged in the future. (https://etherpad.openstack.org/p/modular-l2-agent-outline) On Wed, Jul 9, 2014 at 11:08 PM, Czesnowicz, Przemyslaw mailto:przemyslaw.czesnow...@intel.com>> wrote: Hi We (Intel Openstack team) would like to add support for dpdk based userspace openvswitch using mech_openvswitch and mech_odl from ML2 plugin. The dpdk enabled ovs comes in two flavours one is netdev incorporated into vanilla ovs the other is a fork of ovs with a dpdk datapath (https://github.com/01org/dpdk-ovs ). Both flavours use userspace vhost mechanism to connect the VMs to the switch. Our initial approach was to extend ovs vif bindings in nova and add a config parameter to specify when userspace vhost should be used. Spec : https://review.openstack.org/95805 Code: https://review.openstack.org/100256 Nova devs rejected this approach saying that Neutron should pass all necessary information to nova to select vif bindings. Currently we are looking for a way to pass information from Neutron to Nova that dpdk enabled ovs is being used while still being able to use mech_openvswitch and ovs_neutron_agent or mech_odl. We thought of two possible solutions: 1. Use binding_profile to provide node specific info to nova. Agent rpc api would be extended to allow agents to send node profile to neutron plugin. That info would be stored in db and passed to nova when binding on this specific host is requested. This could be used to support our use case or pass other info to nova (i.e name of integration bridge) 2. Let mech_openvswitch and mech_odl detect what binding type to use. When asked for port binding mech_openvswitch and mech_odl would call the agent or odl to check what bindings to use (VIF_TYPE_OVS or VIF_TYPE_DPDKVHOST) So, what would be the best way to support our usecase, is it one of the above ? Best regards Przemek -- Intel Shannon Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 Business address: Dromore House, East Park, Shannon, Co. Clare This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact th
[openstack-dev] [Neutron][ML2] Support dpdk ovs with ml2 plugin
Hi We (Intel Openstack team) would like to add support for dpdk based userspace openvswitch using mech_openvswitch and mech_odl from ML2 plugin. The dpdk enabled ovs comes in two flavours one is netdev incorporated into vanilla ovs the other is a fork of ovs with a dpdk datapath (https://github.com/01org/dpdk-ovs ). Both flavours use userspace vhost mechanism to connect the VMs to the switch. Our initial approach was to extend ovs vif bindings in nova and add a config parameter to specify when userspace vhost should be used. Spec : https://review.openstack.org/95805 Code: https://review.openstack.org/100256 Nova devs rejected this approach saying that Neutron should pass all necessary information to nova to select vif bindings. Currently we are looking for a way to pass information from Neutron to Nova that dpdk enabled ovs is being used while still being able to use mech_openvswitch and ovs_neutron_agent or mech_odl. We thought of two possible solutions: 1. Use binding_profile to provide node specific info to nova. Agent rpc api would be extended to allow agents to send node profile to neutron plugin. That info would be stored in db and passed to nova when binding on this specific host is requested. This could be used to support our use case or pass other info to nova (i.e name of integration bridge) 2. Let mech_openvswitch and mech_odl detect what binding type to use. When asked for port binding mech_openvswitch and mech_odl would call the agent or odl to check what bindings to use (VIF_TYPE_OVS or VIF_TYPE_DPDKVHOST) So, what would be the best way to support our usecase, is it one of the above ? Best regards Przemek -- Intel Shannon Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 Business address: Dromore House, East Park, Shannon, Co. Clare This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
