On Solaris 11, I see intermittent varnishtest failures on arbitrary tests due to bind failing with EADDRINUSE :
# previous test finishing up ... 22591/1: nanosleep(0xFFFF80FFBFFFE010, 0x00000000) = 0 22591/1: tmout: 0.100000000 sec 22591/1: lwp_wait(2, 0xFFFF80FFBFFFDF9C) = 0 22591/1: lwpid: 2 22591/1: close(3) = 0 22591/1: so_socket(PF_INET, SOCK_STREAM, IPPROTO_TCP, 0, SOV_DEFAULT) = 3 22591/1: setsockopt(3, SOL_SOCKET, SO_REUSEADDR, 0xFFFF80FFBFFFDFA8, 4, SOV_DEFAULT) = 0 22591/1: bind(3, 0x004B8660, 16, SOV_SOCKBSD) Err#125 EADDRINUSE 22591/1: AF_INET name = 127.0.0.1 port = 45199 22591/1: write(2, " b i n d ( )", 6) = 6 22591/1: write(2, " : ", 2) = 2 22578: pollsys(0x00431F80, 1, 0xFFFF80FFBFFFF7E0, 0x00000000) = 1 22578: fd=3 ev=POLLIN|POLLERR|POLLHUP rev=POLLIN 22578: timeout: 59.999000000 sec 22591/1: write(2, 0xFFFF80FFBF48A4CE, 22) = 22 22591/1: A d d r e s s a l r e a d y i n u s e 22591/1: write(2, "\n", 1) = 1 ... The problem can be avoided by not only setting SO_REUSEADDR, but also SO_REUSEPORT in VCC_bind as shown in the attached patch. According to http://lwn.net/Articles/542629/, this could also help performance on huge multiple-acceptor-thread-setups: the SO_REUSEPORT implementation distributes connections evenly across all of the threads (or processes) that are blocked in accept() on the same port. Nils
>From 4c031a611bf6e983eaa5bf55aafd4b0421a996c0 Mon Sep 17 00:00:00 2001 From: Nils Goroll <[email protected]> Date: Mon, 5 May 2014 19:18:52 +0200 Subject: [PATCH 4/4] =?UTF-8?q?use=20SO=5FREUSEPORT=20where=20availab?= =?UTF-8?q?=C3=B6e?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- configure.ac | 8 ++++++++ lib/libvarnish/vss.c | 7 +++++++ 2 files changed, 15 insertions(+) diff --git a/configure.ac b/configure.ac index 36cc542..e8246e0 100644 --- a/configure.ac +++ b/configure.ac @@ -331,6 +331,14 @@ AC_CHECK_DECL([SO_ACCEPTFILTER], #include <sys/socket.h> ] ) +AC_CHECK_DECL([SO_REUSEPORT], + AC_DEFINE(HAVE_SO_REUSEPORT,1,[Define to 1 if you have SO_REUSEPORT]), + , + [ +#include <sys/types.h> +#include <sys/socket.h> + ] +) # Older Solaris versions define SO_{RCV,SND}TIMEO, but do not # implement them. diff --git a/lib/libvarnish/vss.c b/lib/libvarnish/vss.c index 85b3d67..bc4cc77 100644 --- a/lib/libvarnish/vss.c +++ b/lib/libvarnish/vss.c @@ -212,6 +212,13 @@ VSS_bind(const struct vss_addr *va) (void)close(sd); return (-1); } +#ifdef HAVE_SO_REUSEPORT + if (setsockopt(sd, SOL_SOCKET, SO_REUSEPORT, &val, sizeof val) != 0) { + perror("setsockopt(SO_REUSEPORT, 1)"); + (void)close(sd); + return (-1); + } +#endif #ifdef IPV6_V6ONLY /* forcibly use separate sockets for IPv4 and IPv6 */ val = 1; -- 2.0.0.rc0
_______________________________________________ varnish-dev mailing list [email protected] https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
