Open vSwitch userspace can set up flows at a high rate, but it is somewhat
"bursty" in opportunities to set up flows, by which I mean that OVS sets up
a batch of flows, then goes off and does some other work for a while, then
sets up another batch of flows, and so on. The result is that, if a large
number of packets that need flow setups come in all at once, then some of
them can overflow the relatively small kernel-to-user buffers.
This commit increases the kernel-to-user buffers from the default of
approximately 120 kB each to 1 MB each. In one somewhat synthetic test
case that I ran based on an "hping3" that generated a load of about 20,000
new flows per second (including both requests and replies), this reduced
the packets dropped at the kernel-to-user interface from about 30% to none.
I expect that it will similarly improve packet loss in workloads where
flow arrival is not easily predictable.
(This has little effect on workloads generated by "ovs-benchmark rate"
because that benchmark is effectively "self-clocking", that is, a new flow
is triggered only by a reply to a request made earlier, which means that
the number of buffered packets at any given has a known, constant upper
limit.)
Bug #10210.
Signed-off-by: Ben Pfaff <b...@nicira.com>
---
lib/netlink-socket.c | 15 +++++++++++++--
1 files changed, 13 insertions(+), 2 deletions(-)
diff --git a/lib/netlink-socket.c b/lib/netlink-socket.c
index bc46235..7ca44f3 100644
--- a/lib/netlink-socket.c
+++ b/lib/netlink-socket.c
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2008, 2009, 2010, 2011 Nicira Networks.
+ * Copyright (c) 2008, 2009, 2010, 2011, 2012 Nicira Networks.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -44,10 +44,13 @@ COVERAGE_DEFINE(netlink_recv_jumbo);
COVERAGE_DEFINE(netlink_send);
COVERAGE_DEFINE(netlink_sent);
-/* Linux header file confusion causes this to be undefined. */
+/* Linux header file confusion causes these to be undefined. */
#ifndef SOL_NETLINK
#define SOL_NETLINK 270
#endif
+#ifndef SO_RCVBUFFORCE
+#define SO_RCVBUFFORCE 33
+#endif
/* A single (bad) Netlink message can in theory dump out many, many log
* messages, so the burst size is set quite high here to avoid missing useful
@@ -89,6 +92,7 @@ nl_sock_create(int protocol, struct nl_sock **sockp)
struct nl_sock *sock;
struct sockaddr_nl local, remote;
socklen_t local_size;
+ int rcvbuf;
int retval = 0;
if (!max_iovs) {
@@ -122,6 +126,13 @@ nl_sock_create(int protocol, struct nl_sock **sockp)
sock->protocol = protocol;
sock->dump = NULL;
+ rcvbuf = 1024 * 1024;
+ if (setsockopt(sock->fd, SOL_SOCKET, SO_RCVBUFFORCE,
+ &rcvbuf, sizeof rcvbuf)) {
+ VLOG_WARN_RL(&rl, "setting %d-byte socket receive buffer failed (%s)",
+ rcvbuf, strerror(errno));
+ }
+
retval = get_socket_rcvbuf(sock->fd);
if (retval < 0) {
retval = -retval;
--
1.7.2.5
_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev
