On 05/07/2014 11:15 AM, Willy Tarreau wrote:
On Wed, May 07, 2014 at 10:28:18AM +0200, John-Paul Bader wrote:
Willy Tarreau wrote:
It's very interesting, it contains a call to ssl_update_cache(). I didn't
know you were using SSL, but in multi-process mode we have the shared
context
model to share the SSL sessions between processes.
Yes, sorry. In the initial email on this thread I posted our
configuration which included the SSL setup.
Yes I remember having seen your config, but unfortunately I'm having
a hard time remembering all the configs I see during a single day, I'm
sorry.
We're using OpenSSL 1.0.1g 7 Apr 2014 to benefit from the AES-NI
acceleration.
OK.
Oh and BTW, I can confirm that ktrace is really poor compared to strace :-)
haproxy does not include DTrace probes by any chance right? :)
No, and I have no idea how this works either. But if you feel like it
can provide some value and be done without too much effort, feel free
to try :-)
So it seems unrelated to kqueue as well. Later I will try to run the
test with a fraction of the traffic without nbproc (all the traffic is
too much for one process)
That would be great! You can try to build with "USE_PRIVATE_CACHE=1" in
order to disable session sharing.
Right now I'm running a test just with disabled nbproc. Next I will try
to recompile with "USE_PRIVATE_CACHE=1"
Great.
Do I have to pass that option like this:
make CFLAGS="-g -O0" USE_PRIVATE_CACHE=1 ?
Yes that's the principle. You can look at the makefile, all build options
are referenced at the top.
These are our current build options - for completeness:
haproxy -vv
HA-Proxy version 1.5-dev24-8860dcd 2014/04/26
BTW, be careful, a few bugs introduced in dev23 on ACLs were fixed after dev24.
So with this version, "acl foo xxx -i yyy" will not work for example. Balance
url_param is broken as well. All of them are fixed in latest snapshot though.
Copyright 2000-2014 Willy Tarreau <[email protected]>
Build options :
TARGET = freebsd
CPU = generic
CC = cc
CFLAGS = -g -O0 -DFREEBSD_PORTS
OPTIONS = USE_GETADDRINFO=1 USE_ZLIB=1 USE_OPENSSL=1 USE_PCRE=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200
Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.7
Compression algorithms supported : identity, deflate, gzip
Built with OpenSSL version : OpenSSL 1.0.1g 7 Apr 2014
Running on OpenSSL version : OpenSSL 1.0.1g 7 Apr 2014
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 8.34 2013-12-15
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with transparent proxy support using: IP_BINDANY IPV6_BINDANY
Available polling systems :
kqueue : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use kqueue.
OK, nothing unusual here. Thanks for the detailed output, it always
helps!
Best regards,
Willy
Hi All,
I suspect FreeBSD to not support process shared mutex (supported in both
linux and solaris).
I've just made a patch to add errors check on mutex init, and to
fallback on SSL private session cache in error case.
Could you try this patch and feedback to tell us if this warning appear:
"Unable to init lock for the shared SSL session cache. fallback to
private cache."
Regards,
Emeric
>From 69c7942c782ca22180ddbf5679c9c1e045693ff3 Mon Sep 17 00:00:00 2001
From: Emeric Brun <[email protected]>
Date: Wed, 7 May 2014 16:10:18 +0200
Subject: [PATCH] BUG/MAJOR: ssl: Fallback to private session cache if current
lock mode is not supported.
Process shared mutex seems not supported on some OSs (FreeBSD).
This patch checks errors on mutex lock init to fallback
on a private session cache (per process cache) in error cases.
---
include/proto/shctx.h | 3 +++
src/cfgparse.c | 18 ++++++++++++++----
src/shctx.c | 24 ++++++++++++++++++++----
3 files changed, 37 insertions(+), 8 deletions(-)
diff --git a/include/proto/shctx.h b/include/proto/shctx.h
index a84e4a6..e0c695d 100644
--- a/include/proto/shctx.h
+++ b/include/proto/shctx.h
@@ -28,6 +28,9 @@
#define SHCTX_APPNAME "haproxy"
#endif
+#define SHCTX_E_ALLOC_CACHE -1
+#define SHCTX_E_INIT_LOCK -2
+
/* Allocate shared memory context.
* <size> is the number of allocated blocks into cache (default 128 bytes)
* A block is large enough to contain a classic session (without client cert)
diff --git a/src/cfgparse.c b/src/cfgparse.c
index c4f092f..f2f55ed 100644
--- a/src/cfgparse.c
+++ b/src/cfgparse.c
@@ -6744,6 +6744,8 @@ out_uri_auth_compat:
* remains NULL so that listeners can later detach.
*/
list_for_each_entry(bind_conf, &curproxy->conf.bind, by_fe) {
+ int alloc_ctx;
+
if (!bind_conf->is_ssl) {
if (bind_conf->default_ctx) {
Warning("Proxy '%s': A certificate was specified but SSL was not enabled on bind '%s' at [%s:%d] (use 'ssl').\n",
@@ -6758,10 +6760,18 @@ out_uri_auth_compat:
continue;
}
- if (shared_context_init(global.tune.sslcachesize, (global.nbproc > 1) ? 1 : 0) < 0) {
- Alert("Unable to allocate SSL session cache.\n");
- cfgerr++;
- continue;
+ alloc_ctx = shared_context_init(global.tune.sslcachesize, (global.nbproc > 1) ? 1 : 0);
+ if (alloc_ctx < 0) {
+ if (alloc_ctx == SHCTX_E_INIT_LOCK) {
+ Warning("Unable to init lock for the shared SSL session cache. fallback to private cache.\n");
+ alloc_ctx = shared_context_init(global.tune.sslcachesize, 0);
+ }
+
+ if (alloc_ctx < 0) {
+ Alert("Unable to allocate SSL session cache.\n");
+ cfgerr++;
+ continue;
+ }
}
/* initialize all certificate contexts */
diff --git a/src/shctx.c b/src/shctx.c
index f259b9c..6f657a5 100644
--- a/src/shctx.c
+++ b/src/shctx.c
@@ -532,16 +532,32 @@ int shared_context_init(int size, int shared)
PROT_READ | PROT_WRITE, maptype | MAP_ANON, -1, 0);
if (!shctx || shctx == MAP_FAILED) {
shctx = NULL;
- return -1;
+ return SHCTX_E_ALLOC_CACHE;
}
#ifndef USE_PRIVATE_CACHE
#ifdef USE_SYSCALL_FUTEX
shctx->waiters = 0;
#else
- pthread_mutexattr_init(&attr);
- pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
- pthread_mutex_init(&shctx->mutex, &attr);
+ if (pthread_mutexattr_init(&attr)) {
+ munmap(shctx, sizeof(struct shared_context)+(size*sizeof(struct shared_block)));
+ shctx = NULL;
+ return SHCTX_E_INIT_LOCK;
+ }
+
+ if (pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED)) {
+ pthread_mutexattr_destroy(&attr);
+ munmap(shctx, sizeof(struct shared_context)+(size*sizeof(struct shared_block)));
+ shctx = NULL;
+ return SHCTX_E_INIT_LOCK;
+ }
+
+ if (pthread_mutex_init(&shctx->mutex, &attr)) {
+ pthread_mutexattr_destroy(&attr);
+ munmap(shctx, sizeof(struct shared_context)+(size*sizeof(struct shared_block)));
+ shctx = NULL;
+ return SHCTX_E_INIT_LOCK;
+ }
#endif
if (maptype == MAP_SHARED)
use_shared_mem = 1;
--
1.9.1
