Re: [ovs-dev] OVS-Hyper-V-Discovery-Agent Design Document
Hi Alin, Sorry for the late response. I somehow that the message slip by. How do you use vswitchd/netlink to watch VIF creation/deletion and Hyper-V switch changes? Note that through WMI, we monitor the following events: 1. VIF creation/deletion. 2. Hyper-V switch creation/deletion. 3. OVS extension enable/disable on a Hyper-V switch. Also, note that we only register a callback with WMI so that we are passively notified when a change we are interested in happens. We do not run WMI calls in a loop. If you have a better solution, do you mind elaborate a little more on your idea? Best regards, Yin Lin On Mon, Dec 5, 2016 at 12:05 PM, Alin Serdean < aserd...@cloudbasesolutions.com> wrote: > Sorry for the late reply. We had a few days of bank holiday last week. > > > -Original Message- > > From: Ben Pfaff [mailto:b...@ovn.org] > > Sent: Tuesday, November 29, 2016 2:28 AM > > To: Nithin Raju <nit...@vmware.com> > > Cc: Yin Lin <yinli...@gmail.com>; d...@openvswitch.org; Alin Serdean > > <aserd...@cloudbasesolutions.com>; Justin Pettit <jpet...@ovn.org> > > Subject: Re: [ovs-dev] OVS-Hyper-V-Discovery-Agent Design Document > > > > OK, I understand now. > > > > Having ovs-vswitchd itself add ports would be unprecedented. Normally, > we > > depend on some part of the system integration to do that: libvirt does > it in > > modern KVM environments (as you say), a hook script does it on XenServer, > > and so on. > [Alin Serdean] I think we need to look at a bigger picture on what we are > trying to achieve. > AFAIK, in the case of libvirt/Xen, someone asks them to create an adapter > and after it is created the result is added to OVSDB. > On Windows, someone asks vmms (https://technet.microsoft. > com/en-us/library/dd582295(v=ws.10).aspx) to create a port on a switch, > rename the port which was created, and after add it to OVSDB afterwards. > In the case of Windows+OpenStack (with or without OVN) things are already > handled since the code for port renaming is already there. If someone wants > to integrate it in his solution, he could use/reuse/implement the code in > the powershell script which is in our repository (1). > This daemon is targeted for unexperienced users which do not want/do not > know how to do the extra step of renaming the port name. This will give us > better user experience. > The reasons I would like to add the functionality in vswitchd are > simplicity and speed (we see the port creation in the windows datapath, > after it was created by vmms, and during an upcall we could add the port). > > > > My preference would be to keep these details of the system integration > > separate from ovs-vswitchd, since it matches the implementation > > elsewhere. I'd expect this to be a pretty simple daemon, which probably > > wouldn't use much CPU or memory. > > > [Alin Serdean] My main problem with the current implementation is that WMI > calls are slow. Using vswitchd or another monitor that reuses the netlink > implementation would be better IMO. > > (1) https://github.com/openvswitch/ovs/blob/master/ > datapath-windows/misc/OVS.psm1 > > ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] OVS-Hyper-V-Discovery-Agent Design Document
Sorry for the late reply. We had a few days of bank holiday last week. > -Original Message- > From: Ben Pfaff [mailto:b...@ovn.org] > Sent: Tuesday, November 29, 2016 2:28 AM > To: Nithin Raju <nit...@vmware.com> > Cc: Yin Lin <yinli...@gmail.com>; d...@openvswitch.org; Alin Serdean > <aserd...@cloudbasesolutions.com>; Justin Pettit <jpet...@ovn.org> > Subject: Re: [ovs-dev] OVS-Hyper-V-Discovery-Agent Design Document > > OK, I understand now. > > Having ovs-vswitchd itself add ports would be unprecedented. Normally, we > depend on some part of the system integration to do that: libvirt does it in > modern KVM environments (as you say), a hook script does it on XenServer, > and so on. [Alin Serdean] I think we need to look at a bigger picture on what we are trying to achieve. AFAIK, in the case of libvirt/Xen, someone asks them to create an adapter and after it is created the result is added to OVSDB. On Windows, someone asks vmms (https://technet.microsoft.com/en-us/library/dd582295(v=ws.10).aspx) to create a port on a switch, rename the port which was created, and after add it to OVSDB afterwards. In the case of Windows+OpenStack (with or without OVN) things are already handled since the code for port renaming is already there. If someone wants to integrate it in his solution, he could use/reuse/implement the code in the powershell script which is in our repository (1). This daemon is targeted for unexperienced users which do not want/do not know how to do the extra step of renaming the port name. This will give us better user experience. The reasons I would like to add the functionality in vswitchd are simplicity and speed (we see the port creation in the windows datapath, after it was created by vmms, and during an upcall we could add the port). > > My preference would be to keep these details of the system integration > separate from ovs-vswitchd, since it matches the implementation > elsewhere. I'd expect this to be a pretty simple daemon, which probably > wouldn't use much CPU or memory. > [Alin Serdean] My main problem with the current implementation is that WMI calls are slow. Using vswitchd or another monitor that reuses the netlink implementation would be better IMO. (1) https://github.com/openvswitch/ovs/blob/master/datapath-windows/misc/OVS.psm1 ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] OVS-Hyper-V-Discovery-Agent Design Document
On Tue, Nov 29, 2016 at 02:54:38PM -0800, Yin Lin wrote: > 1. I do not like a new daemon either. But as Nithin and Ben pointed out, > there might be a lot of push back if we make ovs-switchd manipulate ovsdb > by adding and deleting ports. > 2. The cmd calls can easily be done away by creating a wrapper around > ovs-vsctl.c and compile it with my daemon. The problem was that all > functions in ovs-vsctl.c is static and I'm not too happy with copying > almost 90% of the codes over to a new file. We need to figure out the best > way to share the code. But according to feedback from Nithin, it is not > urgent for now and we can post is as a follow up patch. There is precedent for a system-specific daemon in OVS. On XenServer, OVS uses a daemon named ovs-xapi-sync to deal with some system integration issues. It's written in Python and the source code is in xenserver/usr_share_openvswitch_scripts_ovs-xapi-sync. If you really need lots of code from ovs-vsctl in your new daemon, then the right thing to do is to move those functions into a library and then use the library from both places. ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] OVS-Hyper-V-Discovery-Agent Design Document
Ben, The current proposal that Yin has put out calls for a new daemon that monitors VIFs that show up on the native HyperV swtitch, gives them a name and adds it to OVSDB. It does it change the existing workflow. You can think of it as how libvirt integrates with OVS in terms of adding ports to OVSDB. Alin's argument is to integrate this new daemon into ovs-vSwitchd and have ovs-vswitchd add ports to OVSDB when ports show up on the HyperV switch. Is there any precedence to ovs-vswitchd adding ports to OVSDB? Would this be a divergence? Thanks, -- Nithin On Mon, Nov 28, 2016 at 4:00 PM -0800, "Ben Pfaff" <b...@ovn.org<mailto:b...@ovn.org>> wrote: Does this actually call for vswitchd to add ports? Reading it, it looks like the new daemon does that. But I didn't read it carefully enough to understand it in full. On Mon, Nov 28, 2016 at 11:13:21PM +, Nithin Raju wrote: > hi Alin, > Thanks for the comments. > > There has not been a case so far (AFAIK) where vswitchd added ports to OVSDB. > This would be a diversion from that. We’ll have to take this up with Ben or > Justin to see how they feel. > > Thanks, > -- Nithin > > On Nov 28, 2016, at 11:18 AM, Alin Serdean > <aserd...@cloudbasesolutions.com<mailto:aserd...@cloudbasesolutions.com>> > wrote: > > Hi Yin, > > Thanks a lot for the patch and the document! > > There are a few concerns that I have with this implementation: > - the need for a new daemon, > - use of cmd calls to add/delete ports, > - addition of .NET to compile the binaries. > > As previously discussed in the meetings, we could just use 'ovs-vswitchd' > with an argument i.e. '--integration_bridge=' to add the ports. We > see the port creations in the kernel and we could issue an event to the > userspace to add ports under that bridge. We could add a new type of 'action' > or try to reuse the events that are already present in the implementation. > Another advantage to use ovs-vswitchd is that we could talk directly to > ovsdb. Also, no additional dependency needed. > > We could add the argument '--integration_bridge=' when creating the > service via the installer with a nice text to inform the user about what it > does. Changing the service properties is normal activity for a system > administrator, so he could easily remove it afterwards. > > Thanks, > Alin. > > -Original Message- > From: ovs-dev-boun...@openvswitch.org<mailto:ovs-dev-boun...@openvswitch.org> > [mailto:ovs-dev- > boun...@openvswitch.org<mailto:boun...@openvswitch.org>] On Behalf Of Yin Lin > Sent: Monday, November 21, 2016 10:07 PM > To: d...@openvswitch.org<mailto:d...@openvswitch.org> > Subject: [ovs-dev] OVS-Hyper-V-Discovery-Agent Design Document > > Hi, > > Below is a document that describes the design and implementation of VIF > discovery agent for Hyper-V that I have been working on. The coding has > already been completed and will be sent out for review in a follow up patch. > The document describes the effort and the choices made. Thanks Sairam > Venugopal for the initial review and edit of the document > > > > Please have a look, and get in touch for any questions or comments. > > > > Thanks! > > -- Yin > ___ > dev mailing list > d...@openvswitch.org<mailto:d...@openvswitch.org> > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev=DgICAg=uilaK90D4TOVoH58JNXRgQ=-xl6DPE_Y3uQD-mpZD7osBo2iL4s3jwdmSjTlGgjlsQ=GMr-AZul4UsNIDzghSvQVWBWWf9ZXBYbRf5TFXxEtFg=tAcMGwbQ11t617kkksED_MalGT4y_bWtSDojOFp58vc= > ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] OVS-Hyper-V-Discovery-Agent Design Document
Does this actually call for vswitchd to add ports? Reading it, it looks like the new daemon does that. But I didn't read it carefully enough to understand it in full. On Mon, Nov 28, 2016 at 11:13:21PM +, Nithin Raju wrote: > hi Alin, > Thanks for the comments. > > There has not been a case so far (AFAIK) where vswitchd added ports to OVSDB. > This would be a diversion from that. We’ll have to take this up with Ben or > Justin to see how they feel. > > Thanks, > -- Nithin > > On Nov 28, 2016, at 11:18 AM, Alin Serdean > <aserd...@cloudbasesolutions.com<mailto:aserd...@cloudbasesolutions.com>> > wrote: > > Hi Yin, > > Thanks a lot for the patch and the document! > > There are a few concerns that I have with this implementation: > - the need for a new daemon, > - use of cmd calls to add/delete ports, > - addition of .NET to compile the binaries. > > As previously discussed in the meetings, we could just use 'ovs-vswitchd' > with an argument i.e. '--integration_bridge=' to add the ports. We > see the port creations in the kernel and we could issue an event to the > userspace to add ports under that bridge. We could add a new type of 'action' > or try to reuse the events that are already present in the implementation. > Another advantage to use ovs-vswitchd is that we could talk directly to > ovsdb. Also, no additional dependency needed. > > We could add the argument '--integration_bridge=' when creating the > service via the installer with a nice text to inform the user about what it > does. Changing the service properties is normal activity for a system > administrator, so he could easily remove it afterwards. > > Thanks, > Alin. > > -Original Message- > From: ovs-dev-boun...@openvswitch.org<mailto:ovs-dev-boun...@openvswitch.org> > [mailto:ovs-dev- > boun...@openvswitch.org<mailto:boun...@openvswitch.org>] On Behalf Of Yin Lin > Sent: Monday, November 21, 2016 10:07 PM > To: d...@openvswitch.org<mailto:d...@openvswitch.org> > Subject: [ovs-dev] OVS-Hyper-V-Discovery-Agent Design Document > > Hi, > > Below is a document that describes the design and implementation of VIF > discovery agent for Hyper-V that I have been working on. The coding has > already been completed and will be sent out for review in a follow up patch. > The document describes the effort and the choices made. Thanks Sairam > Venugopal for the initial review and edit of the document > > > > Please have a look, and get in touch for any questions or comments. > > > > Thanks! > > -- Yin > ___ > dev mailing list > d...@openvswitch.org<mailto:d...@openvswitch.org> > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev=DgICAg=uilaK90D4TOVoH58JNXRgQ=-xl6DPE_Y3uQD-mpZD7osBo2iL4s3jwdmSjTlGgjlsQ=GMr-AZul4UsNIDzghSvQVWBWWf9ZXBYbRf5TFXxEtFg=tAcMGwbQ11t617kkksED_MalGT4y_bWtSDojOFp58vc= > ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] OVS-Hyper-V-Discovery-Agent Design Document
hi Alin, Thanks for the comments. There has not been a case so far (AFAIK) where vswitchd added ports to OVSDB. This would be a diversion from that. We’ll have to take this up with Ben or Justin to see how they feel. Thanks, -- Nithin On Nov 28, 2016, at 11:18 AM, Alin Serdean <aserd...@cloudbasesolutions.com<mailto:aserd...@cloudbasesolutions.com>> wrote: Hi Yin, Thanks a lot for the patch and the document! There are a few concerns that I have with this implementation: - the need for a new daemon, - use of cmd calls to add/delete ports, - addition of .NET to compile the binaries. As previously discussed in the meetings, we could just use 'ovs-vswitchd' with an argument i.e. '--integration_bridge=' to add the ports. We see the port creations in the kernel and we could issue an event to the userspace to add ports under that bridge. We could add a new type of 'action' or try to reuse the events that are already present in the implementation. Another advantage to use ovs-vswitchd is that we could talk directly to ovsdb. Also, no additional dependency needed. We could add the argument '--integration_bridge=' when creating the service via the installer with a nice text to inform the user about what it does. Changing the service properties is normal activity for a system administrator, so he could easily remove it afterwards. Thanks, Alin. -Original Message- From: ovs-dev-boun...@openvswitch.org<mailto:ovs-dev-boun...@openvswitch.org> [mailto:ovs-dev- boun...@openvswitch.org<mailto:boun...@openvswitch.org>] On Behalf Of Yin Lin Sent: Monday, November 21, 2016 10:07 PM To: d...@openvswitch.org<mailto:d...@openvswitch.org> Subject: [ovs-dev] OVS-Hyper-V-Discovery-Agent Design Document Hi, Below is a document that describes the design and implementation of VIF discovery agent for Hyper-V that I have been working on. The coding has already been completed and will be sent out for review in a follow up patch. The document describes the effort and the choices made. Thanks Sairam Venugopal for the initial review and edit of the document Please have a look, and get in touch for any questions or comments. Thanks! -- Yin ___ dev mailing list d...@openvswitch.org<mailto:d...@openvswitch.org> https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev=DgICAg=uilaK90D4TOVoH58JNXRgQ=-xl6DPE_Y3uQD-mpZD7osBo2iL4s3jwdmSjTlGgjlsQ=GMr-AZul4UsNIDzghSvQVWBWWf9ZXBYbRf5TFXxEtFg=tAcMGwbQ11t617kkksED_MalGT4y_bWtSDojOFp58vc= ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] OVS-Hyper-V-Discovery-Agent Design Document
On Mon, Nov 21, 2016 at 12:06:49PM -0800, Yin Lin wrote: > Below is a document that describes the design and implementation of VIF > discovery agent for Hyper-V that I have been working on. The coding has > already been completed and will be sent out for review in a follow up > patch. The document describes the effort and the choices made. Thanks > Sairam Venugopal for the initial review and edit of the document > > > > Please have a look, and get in touch for any questions or comments. This seems like a pretty comprehensive document. I hope that it will be included in the source tree as part of the patch that implements the feature. I don't have anything to say about the content itself. I'll leave that for folks familiar with Hyper-V to judge. ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
[ovs-dev] OVS-Hyper-V-Discovery-Agent Design Document
Hi, Below is a document that describes the design and implementation of VIF discovery agent for Hyper-V that I have been working on. The coding has already been completed and will be sent out for review in a follow up patch. The document describes the effort and the choices made. Thanks Sairam Venugopal for the initial review and edit of the document Please have a look, and get in touch for any questions or comments. Thanks! -- Yin === OVS-Hyper-V-Discovery-Agent Design Document == On Hyper-V, virtual switches and virtual adapters are created and managed by Windows infrastructure. This is currently not integrated with OVS. When a new virtual adapter is connected to a virtual switch with OVS extension enabled, users need to manually tag it with an OVS port name and manually use OVS userspace utilities such as ovs-vsctl to add a corresponding OVS port. This document describes design of a VIF discovery agent for OVS on Hyper-V that functions as a counterpart to libvirt on KVM. The agent monitors virtual switch/adapter modifications, tags the adapter and adds/deletes OVS port as necessary to automate the VIF creation/deletion process. The rest of this document describes: 1. Background of OVS port management on Hyper-V 2. Discovery agent functional specification 3. Implementation of discovery agent 4. Build/Deployment environment For more questions, please contact dev at openvswitch.org 1) Background of OVS port management on Hyper-V OVS kernel module on Hyper-V[3] is implemented as a extension [2] to the virtual extensible switch[1] provided by Microsoft. The extension can be enabled through Hyper-V Virtual Switch Manager GUI or Powershell command once the OVS driver is installed. The OVS kernel module recognizes any adapter that connects to such a switch by requiring user to tag the Hyper-V adapter with an OVS port name and then add the OVS port to ovsdb with ovs-vsctl command. A Powershell module called OVS.psm1 has been implemented to facilitate the tagging. Once the module is imported, admin can use the following command to tag a virtual adapter with OVS port name: $vnic | Set-VMNetworkAdapterOVSPort -OVSPortName $vifName where $vnic is a VMNetworkAdapter instance and $vifName is the OVS port name. Then the admin needs to add the OVS port by using: ovs-vsctl add-port $bridge $vifName --set interface $vifName external_ids:vm-id=$vmId \ external_ids:iface-status=active where $bridge is the integration bridge name and $vmId is the Hyper-V VM UUID. The VM ID is currently optional for reporting purposes and will not affect the actual functionality of the OVS port. In order to automate this process, the discovery agent needs to actively monitor two events: 1. Creation/Deletion of a virtual switch that has OVS extension enabled. This includes enable/disable-ment of OVS extension on an existing virtual switch. 2. Connection/Disconnection of a VIF to OVS enabled virtual switch. In order to capture these events, we utilize Windows Management Instrumentation (WMI)[4] to register a callback [5] when a desired change in virtualization namespace happens. WMI is also used to perform a full sync of all existing virtual switches and virtual adapters during the discovery agent's initial boot. 2) Discovery agent functional specification --- The discovery agent performs the following three basic duties: 1. Tag(). Tag a virtual adapter with OVS port, by setting its "ElementName" field in the WMI table "Msvm_EthernetPortAllocationSettingData" to an OVS port name, such that it can be picked up by OVS kernel module and matched against a corresponding entry in ovsdb. By default, the OVS port name of a virtual adapter assigned by the discovery agent is the same as its UUID assigned by Windows. 2. AddPort(). Add an OVS port to OVS bridge. If a virtual adapter is connected to an OVS switch but it's missing in ovsdb, the daemon adds it to a preconfigured integration bridge, as specified by the admin.. 3. DelPort(). Delete an OVS port from the OVS integration bridge. When the discovery agent starts, it scans all virtual adapters that are connected to the virtual switch with OVS extension enabled. , It runs AddPort() for adapters that are missing in the preconfigured OVS integration bridge. After that, the discovery agent actively monitors the system for any relevant virtual switch/adapter changes and performs the following actions: 1. When OVS extension is enabled on a virtual switch, perform AddPort() for all adapters connected to the virtual switch. 2. When a virtual switch with OVS extension is deleted, or when OVS extension is disabled on a virtual switch, perform DelPort() for all adapters previously connected to the virtual switch. 3. When a virtual adapter is created
