On 2020/05/05 1:24, William Tu wrote:
On Tue, Apr 21, 2020 at 11:47:00PM +0900, Toshiaki Makita wrote:
This patch adds an XDP-based flow cache using the OVS netdev-offload
flow API provider. When an OVS device with XDP offload enabled,
packets first are processed in the XDP flow cache (with parse, and
table lookup implemented in eBPF) and if hits, the action processing
are also done in the context of XDP, which has the minimum overhead.
This provider is based on top of William's recently posted patch for
custom XDP load. When a custom XDP is loaded, the provider detects if
the program supports classifier, and if supported it starts offloading
flows to the XDP program.
The patches are derived from xdp_flow[1], which is a mechanism similar to
this but implemented in kernel.
* Motivation
While userspace datapath using netdev-afxdp or netdev-dpdk shows good
performance, there are use cases where packets better to be processed in
kernel, for example, TCP/IP connections, or container to container
connections. Current solution is to use tap device or af_packet with
extra kernel-to/from-userspace overhead. But with XDP, a better solution
is to steer packets earlier in the XDP program, and decides to send to
userspace datapath or stay in kernel.
One problem with current netdev-afxdp is that it forwards all packets to
userspace, The first patch from William (netdev-afxdp: Enable loading XDP
program.) only provides the interface to load XDP program, howerver users
usually don't know how to write their own XDP program.
XDP also supports HW-offload so it may be possible to offload flows to
HW through this provider in the future, although not currently.
The reason is that map-in-map is required for our program to support
classifier with subtables in XDP, but map-in-map is not offloadable.
If map-in-map becomes offloadable, HW-offload of our program will also
be doable.
* How to use
1. Install clang/llvm >= 9, libbpf >= 0.0.4, and kernel >= 5.3.
2. make with --enable-afxdp --enable-bpf
--enable-bpf will generate XDP program "bpf/flowtable_afxdp.o". Note that
the BPF object will not be installed anywhere by "make install" at this point.
When configure, there is a missing include, causing error due to __u64
checking bpf/bpf_helpers.h usability... no
checking bpf/bpf_helpers.h presence... yes
configure: WARNING: bpf/bpf_helpers.h: present but cannot be compiled
configure: WARNING: bpf/bpf_helpers.h: check for missing prerequisite
headers?
configure:18876: gcc -c -g -O2 conftest.c >&5
In file included from /usr/local/include/bpf/bpf_helpers.h:5:0,
from conftest.c:73:
/usr/local/include/bpf/bpf_helper_defs.h:55:82: error: unknown type name '__u64'
static int (*bpf_map_update_elem)(void *map, const void *key, const void
*value, __u64 flags) = (void *) 2;
^
/usr/local/include/bpf/bpf_helper_defs.h:79:41: error: unknown type name '__u32'
static int (*bpf_probe_read)(void *dst, __u32 size, const void *unsafe_ptr) =
(void *) 4;
I applied this to fix it:
diff --git a/acinclude.m4 b/acinclude.m4
index 5eeab6feb9cc..39dfce565182 100644
--- a/acinclude.m4
+++ b/acinclude.m4
@@ -326,7 +326,8 @@ AC_DEFUN([OVS_CHECK_LINUX_BPF], [
[AC_MSG_ERROR([unable to find llc to compile BPF program])])
AC_CHECK_HEADER([bpf/bpf_helpers.h], [],
- [AC_MSG_ERROR([unable to find bpf/bpf_helpers.h to compile BPF
program])])
+ [AC_MSG_ERROR([unable to find bpf/bpf_helpers.h to compile BPF
program])],
+ [#include <linux/types.h>])
AC_CHECK_HEADER([linux/bpf.h], [],
[AC_MSG_ERROR([unable to find linux/bpf.h to compile BPF program])])
Oh thanks, not sure why I didn't hit this problem.
3. Load custom XDP program
E.g.
$ ovs-vsctl add-port ovsbr0 veth0 -- set int veth0 options:xdp-mode=native \
options:xdp-obj="path/to/ovs/bpf/flowtable_afxdp.o"
$ ovs-vsctl add-port ovsbr0 veth1 -- set int veth1 options:xdp-mode=native \
options:xdp-obj="path/to/ovs/bpf/flowtable_afxdp.o"
4. Enable XDP_REDIRECT
If you use veth devices, make sure to load some (possibly dummy) programs
on the peers of veth devices.
5. Enable hw-offload
$ ovs-vsctl set Open_vSwitch . other_config:offload-driver=linux_xdp
$ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
This will starts offloading flows to the XDP program.
Thanks, I can succefully get it working...
When applying your patch on current master, I hit a bug.
I will send a patch to fix it.
diff --git a/lib/netdev-linux.c b/lib/netdev-linux.c
index 40d0cc1105ea..b52071e92ec7 100644
--- a/lib/netdev-linux.c
+++ b/lib/netdev-linux.c
@@ -3588,6 +3588,7 @@ const struct netdev_class netdev_internal_class = {
#ifdef HAVE_AF_XDP
#define NETDEV_AFXDP_CLASS_COMMON \
+ .init = netdev_afxdp_init, \
.construct = netdev_afxdp_construct, \
.destruct = netdev_afxdp_destruct, \
.get_stats = netdev_afxdp_get_stats, \
Seem you have already fixed it. Thanks.
Other part works fine.
I'm planning to play with more rules and performance.
Thank you for trying!
Toshiaki Makita
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
