> On 4/11/2019 10:48 AM, Nikolas Britton wrote: >> I'm not sure what you mean by pulled from the OOT drivers. I was able >> to build the kernel modules with a few hacks to ./debian/rules and >> everything necessary was already there. I was able to build the >> openvswitch.ko, vport_geneve.ko, vport_vxlan.ko, vport_gre.ko, >> vport_lisp.ko, and vport_stt.ko with dpdk appending the following in >> ./debian/rules for configure: >> >> --with-dpdk=/usr/src/dpdk-stable-18.11.1/x86_64-native-linuxapp-gcc >> --with-linux=/usr/src/linux-headers-4.18.0-17-generic > > Top posting... so hard to follow. > > In any case I misunderstood - I was referring to the Linux upstream > kernel where the stt and lisp drivers are > not included. If Ubuntu adds them to their distro package then that > changes the calculus. > > So now I'm confused though - the stt driver is in the native Linux > kernel datapath but you're using dpdk to > bypass the kernel datapath? > > Maybe someone who knows DPDK better can help.
Hi. Userspace has no STT support. So, there is no any need to build OVS with DPDK. '--with-dpdk' option could be simply omitted. And there is no real need to rebuild the OVS userspace components, only the 'vport-stt' kernel module. Best regards, Ilya Maximets. > > - Greg > >> >> For some reason the dpdk 18.11 upstream debian packages doesn't build >> / install the libraries necessary for the openvswitch configure script >> to use on ubuntu 18.04, so I had to first manually build the dpdk >> libraries by running make install T=x86_64-native-linuxapp-gcc in >> /usr/src/dpdk-stable-18.11.1/. More information about that process is >> documented here: http://docs.openvswitch.org/en/latest/intro/install/dpdk/ >> >> I compiled it for skylake-avx512, here are my flags: >> export DEB_CFLAGS_APPEND="-m64 -Ofast -march=skylake-avx512 >> -mtune=skylake-avx512 -funroll-loops" >> export DEB_CXXFLAGS_APPEND="-m64 -Ofast -march=skylake-avx512 >> -mtune=skylake-avx512 -funroll-loops" >> export DEB_LDFLAGS_APPEND="-m64 -Ofast -march=skylake-avx512 >> -mtune=skylake-avx512 -funroll-loops" >> export DEB_OBJCFLAGS_APPEND="-m64 -Ofast -march=skylake-avx512 >> -mtune=skylake-avx512 -funroll-loops" >> export DEB_OBJCXXFLAGS_APPEND="-m64 -Ofast -march=skylake-avx512 >> -mtune=skylake-avx512 -funroll-loops" >> export DEB_CPPFLAGS_APPEND="-m64 -Ofast -march=skylake-avx512 >> -mtune=skylake-avx512 -funroll-loops" >> >> I uploaded all the packages to here: http://www.exabit.io/ubuntu/ >> >> I did an initial test with the vport-stt.ko, and openvswitch.ko, >> modules enabled and performance was amazing to say the least, iperf >> test appeared to fully saturate my 16 Gbps connection between two nodes. >> >> root at bm001:~# ovs-vsctl show >> 224dea41-f166-4f33-aa31-372fa5617624 >> Bridge "overlay0" >> Port "stt0" >> Interface "stt0" >> type: stt >> options: {remote_cert="/etc/openvswitch/bm002-cert.pem", >> remote_ip="10.3.40.52"} >> Port "overlay0" >> Interface "overlay0" >> type: internal >> ovs_version: "2.11.1" >> >> root at bm001:~# ifconfig overlay0 >> overlay0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 >> inet 192.168.144.1 netmask 255.255.255.0 broadcast >> 192.168.144.255 >> inet6 fe80::181b:5dff:fed5:bc4f prefixlen 64 scopeid 0x20<link> >> ether 1a:1b:5d:d5:bc:4f txqueuelen 1000 (Ethernet) >> RX packets 755447 bytes 36222552 (36.2 MB) >> RX errors 0 dropped 0 overruns 0 frame 0 >> TX packets 714694 bytes 44007592220 (44.0 GB) >> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 >> >> root at bm001:~# ifconfig ens3 >> ens3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000 >> inet 10.3.40.51 netmask 255.255.255.0 broadcast 10.3.40.255 >> inet6 fe80::17ff:fe00:57ce prefixlen 64 scopeid 0x20<link> >> ether 02:00:17:00:57:ce txqueuelen 1000 (Ethernet) >> RX packets 13310513 bytes 1332012738 (1.3 GB) >> RX errors 0 dropped 1570 overruns 0 frame 0 >> TX packets 20576288 bytes 45912880820 (45.9 GB) >> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 >> >> root at bm001:~# iperf -c 192.168.144.2 >> ------------------------------------------------------------ >> Client connecting to 192.168.144.2, TCP port 5001 >> TCP window size: 85.0 KByte (default) >> ------------------------------------------------------------ >> [ 3] local 192.168.144.1 port 32834 connected with 192.168.144.2 port >> 5001 >> [ ID] Interval Transfer Bandwidth >> [ 3] 0.0-10.0 sec 17.7 GBytes 15.2 Gbits/sec >> >> root at bm002:~# ovs-vsctl show >> 75ef1aaf-a31a-4e87-b3d3-94241d9b7125 >> Bridge "overlay0" >> Port "overlay0" >> Interface "overlay0" >> type: internal >> Port "stt0" >> Interface "stt0" >> type: stt >> options: {local_ip="10.3.40.52", >> remote_cert="/etc/openvswitch/bm001-cert.pem", remote_ip="10.3.40.51"} >> ovs_version: "2.11.1" >> >> overlay0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 >> inet 192.168.144.2 netmask 255.255.255.0 broadcast >> 192.168.144.255 >> inet6 fe80::1097:22ff:fe53:1e49 prefixlen 64 scopeid 0x20<link> >> ether 12:97:22:53:1e:49 txqueuelen 1000 (Ethernet) >> RX packets 944565 bytes 44007205884 (44.0 GB) >> RX errors 0 dropped 0 overruns 0 frame 0 >> TX packets 517923 bytes 34184906 (34.1 MB) >> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 >> >> ens3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000 >> inet 10.3.40.52 netmask 255.255.255.0 broadcast 10.3.40.255 >> inet6 fe80::17ff:fe02:12b8 prefixlen 64 scopeid 0x20<link> >> ether 02:00:17:02:12:b8 txqueuelen 1000 (Ethernet) >> RX packets 20679038 bytes 45910655685 (45.9 GB) >> RX errors 0 dropped 0 overruns 0 frame 0 >> TX packets 13338314 bytes 1344124706 (1.3 GB) >> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 >> >> root at bm002:~# iperf -s -B 192.168.144.2 >> ------------------------------------------------------------ >> Server listening on TCP port 5001 >> Binding to local address 192.168.144.2 >> TCP window size: 85.3 KByte (default) >> ------------------------------------------------------------ >> [ 4] local 192.168.144.2 port 5001 connected with 192.168.144.1 port >> 32834 >> [ ID] Interval Transfer Bandwidth >> [ 4] 0.0-10.0 sec 17.7 GBytes 15.2 Gbits/sec >> >> On Wed, Apr 10, 2019 at 5:53 PM Gregory Rose <gvrose8192 at gmail.com >> <mailto:gvrose8192 at gmail.com>> wrote: >> >> >> >> On 4/10/2019 1:32 PM, Nikolas Britton wrote: >> > Hi, >> > >> > I just finished compiling OVS 2.11.0 packages for Ubuntu 18.04 >> using the >> > Debian upstream source package. I was experimenting with tunnels >> to create >> > an overlay network to connect multiple KVM hypervisors. One of >> the options >> > I tried to use was an STT tunnel. However, it appears that the >> STT protocol >> > didn't get included in the binary somehow. Does it need to be >> enabled with >> > a build flag? The STT option also doesn't work with IPsec, but >> the manual >> > says it should. >> >> The STT driver isn't included in the upstream drivers. You'd need to >> pull from the OOT drivers at >> github. >> >> - Greg >> >> >> >> -- >> *Nikolas Britton* >> *Senior Deployment Engineer, Mirantis* >> 12301-B Riata Trace Pkwy Bldg #2, Suite 150, Austin, Texas _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
