On 4/11/2019 10:48 AM, Nikolas Britton wrote:
I'm not sure what you mean by pulled from the OOT drivers. I was able
to build the kernel modules with a few hacks to ./debian/rules and
everything necessary was already there. I was able to build the
openvswitch.ko, vport_geneve.ko, vport_vxlan.ko, vport_gre.ko,
vport_lisp.ko, and vport_stt.ko with dpdk appending the following in
./debian/rules for configure:
--with-dpdk=/usr/src/dpdk-stable-18.11.1/x86_64-native-linuxapp-gcc
--with-linux=/usr/src/linux-headers-4.18.0-17-generic
Top posting... so hard to follow.
In any case I misunderstood - I was referring to the Linux upstream
kernel where the stt and lisp drivers are
not included. If Ubuntu adds them to their distro package then that
changes the calculus.
So now I'm confused though - the stt driver is in the native Linux
kernel datapath but you're using dpdk to
bypass the kernel datapath?
Maybe someone who knows DPDK better can help.
- Greg
For some reason the dpdk 18.11 upstream debian packages doesn't build
/ install the libraries necessary for the openvswitch configure script
to use on ubuntu 18.04, so I had to first manually build the dpdk
libraries by running make install T=x86_64-native-linuxapp-gcc in
/usr/src/dpdk-stable-18.11.1/. More information about that process is
documented here: http://docs.openvswitch.org/en/latest/intro/install/dpdk/
I compiled it for skylake-avx512, here are my flags:
export DEB_CFLAGS_APPEND="-m64 -Ofast -march=skylake-avx512
-mtune=skylake-avx512 -funroll-loops"
export DEB_CXXFLAGS_APPEND="-m64 -Ofast -march=skylake-avx512
-mtune=skylake-avx512 -funroll-loops"
export DEB_LDFLAGS_APPEND="-m64 -Ofast -march=skylake-avx512
-mtune=skylake-avx512 -funroll-loops"
export DEB_OBJCFLAGS_APPEND="-m64 -Ofast -march=skylake-avx512
-mtune=skylake-avx512 -funroll-loops"
export DEB_OBJCXXFLAGS_APPEND="-m64 -Ofast -march=skylake-avx512
-mtune=skylake-avx512 -funroll-loops"
export DEB_CPPFLAGS_APPEND="-m64 -Ofast -march=skylake-avx512
-mtune=skylake-avx512 -funroll-loops"
I uploaded all the packages to here: http://www.exabit.io/ubuntu/
I did an initial test with the vport-stt.ko, and openvswitch.ko,
modules enabled and performance was amazing to say the least, iperf
test appeared to fully saturate my 16 Gbps connection between two nodes.
root@bm001:~# ovs-vsctl show
224dea41-f166-4f33-aa31-372fa5617624
Bridge "overlay0"
Port "stt0"
Interface "stt0"
type: stt
options: {remote_cert="/etc/openvswitch/bm002-cert.pem",
remote_ip="10.3.40.52"}
Port "overlay0"
Interface "overlay0"
type: internal
ovs_version: "2.11.1"
root@bm001:~# ifconfig overlay0
overlay0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.144.1 netmask 255.255.255.0 broadcast
192.168.144.255
inet6 fe80::181b:5dff:fed5:bc4f prefixlen 64 scopeid 0x20<link>
ether 1a:1b:5d:d5:bc:4f txqueuelen 1000 (Ethernet)
RX packets 755447 bytes 36222552 (36.2 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 714694 bytes 44007592220 (44.0 GB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
root@bm001:~# ifconfig ens3
ens3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000
inet 10.3.40.51 netmask 255.255.255.0 broadcast 10.3.40.255
inet6 fe80::17ff:fe00:57ce prefixlen 64 scopeid 0x20<link>
ether 02:00:17:00:57:ce txqueuelen 1000 (Ethernet)
RX packets 13310513 bytes 1332012738 (1.3 GB)
RX errors 0 dropped 1570 overruns 0 frame 0
TX packets 20576288 bytes 45912880820 (45.9 GB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
root@bm001:~# iperf -c 192.168.144.2
------------------------------------------------------------
Client connecting to 192.168.144.2, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.144.1 port 32834 connected with 192.168.144.2 port
5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 17.7 GBytes 15.2 Gbits/sec
root@bm002:~# ovs-vsctl show
75ef1aaf-a31a-4e87-b3d3-94241d9b7125
Bridge "overlay0"
Port "overlay0"
Interface "overlay0"
type: internal
Port "stt0"
Interface "stt0"
type: stt
options: {local_ip="10.3.40.52",
remote_cert="/etc/openvswitch/bm001-cert.pem", remote_ip="10.3.40.51"}
ovs_version: "2.11.1"
overlay0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.144.2 netmask 255.255.255.0 broadcast
192.168.144.255
inet6 fe80::1097:22ff:fe53:1e49 prefixlen 64 scopeid 0x20<link>
ether 12:97:22:53:1e:49 txqueuelen 1000 (Ethernet)
RX packets 944565 bytes 44007205884 (44.0 GB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 517923 bytes 34184906 (34.1 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000
inet 10.3.40.52 netmask 255.255.255.0 broadcast 10.3.40.255
inet6 fe80::17ff:fe02:12b8 prefixlen 64 scopeid 0x20<link>
ether 02:00:17:02:12:b8 txqueuelen 1000 (Ethernet)
RX packets 20679038 bytes 45910655685 (45.9 GB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 13338314 bytes 1344124706 (1.3 GB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
root@bm002:~# iperf -s -B 192.168.144.2
------------------------------------------------------------
Server listening on TCP port 5001
Binding to local address 192.168.144.2
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[ 4] local 192.168.144.2 port 5001 connected with 192.168.144.1 port
32834
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 17.7 GBytes 15.2 Gbits/sec
On Wed, Apr 10, 2019 at 5:53 PM Gregory Rose <[email protected]
<mailto:[email protected]>> wrote:
On 4/10/2019 1:32 PM, Nikolas Britton wrote:
> Hi,
>
> I just finished compiling OVS 2.11.0 packages for Ubuntu 18.04
using the
> Debian upstream source package. I was experimenting with tunnels
to create
> an overlay network to connect multiple KVM hypervisors. One of
the options
> I tried to use was an STT tunnel. However, it appears that the
STT protocol
> didn't get included in the binary somehow. Does it need to be
enabled with
> a build flag? The STT option also doesn't work with IPsec, but
the manual
> says it should.
The STT driver isn't included in the upstream drivers. You'd need to
pull from the OOT drivers at
github.
- Greg
--
*Nikolas Britton*
*Senior Deployment Engineer, Mirantis*
12301-B Riata Trace Pkwy Bldg #2, Suite 150, Austin, Texas
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
