Re: [ANNOUNCE] haproxy-1.8-rc1 : the last mile

2017-10-31 Thread Cyril Bonté 

Hi all,

Le 01/11/2017 à 00:20, Willy Tarreau a écrit :

[...]
So what do we have here ? A *preview* of what the final 1.8 will be like.
[...]
Please find the usual URLs below :
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


This announcement was exciting enough to take some time to regenerate an 
up to date HTML documentation ! 1.8-rc1 is now available : 
http://cbonte.github.io/haproxy-dconv/1.8/configuration.html



Have fun,
Willy -- feeling exhausted like a marathoner :-)


Great job ! Now it's time to test and track nasty bugs before the final 
1.8 release ;-)



--
Cyril Bonté

Fetch DN according to RFC 2253

2017-10-31 Thread Joao Morais 

Hi list, is there a way to choose between pre and pos RFC 2253[1] format of DN 
from a client cert? Here are nginx[2] and Apache[3] docs about the subject.

[1] https://tools.ietf.org/html/rfc2253
[2] http://nginx.org/en/docs/http/ngx_http_ssl_module.html#var_ssl_client_s_dn
[3] http://httpd.apache.org/docs/2.4/mod/mod_ssl.html#ssloptions (look for 
LegacyDNStringFormat option)

[ANNOUNCE] haproxy-1.8-rc1 : the last mile

2017-10-31 Thread Willy Tarreau 
Hi all!

Give some food, drink, disk space, a quiet place, a keyboard and Git to
a developer and he will code forever... All good things must come to an
end, and I decided that it had to be right now. After all, we initially
announced the end of developments around end of September and a release
around October or November. We've missed the September deadline, but I
know by experience that nobody notices missed deadlines as long as they
are not missed by more than one month. And one month ends today.

So what do we have here ? A *preview* of what the final 1.8 will be like.
As a reminder, our -rc don't mean they're totally ready, but that we're
really done with development and only performing the last adjustments,
fixes, cleanups, documentation updates etc and that NEW FEATURES ARE NOT
WELCOME ANYMORE FOR THIS VERSION. There are still some issues (I found
a few after the release) but overall there's enough in various areas to
satisfy the curious. And we needed to merge our various branches to take
the time to fix conflicts and adapt our respective code (mainly to
threads, which were merged first).

In practice, since 1.8-dev3, the following main features were merged :
  - multi-thread support : for certain workloads like massive SSL rekeying
it provides comparable performance to multi-process, but with all the
load handled in a single process, hence single health checks and server
states, single stats, single CLI etc. There are some known scalability
limitations coming from arbitrations we had to do for 1.8 and which we
will address during 1.9 (and maybe some before 1.8-final). The feature
is enabled by default on linux2628 and freebsd, and disabled by default
on other targets, though it can be forced enabled using USE_THREAD=1 or
forced disabled using USE_THREAD= (empty string). Once enabled, it's
possible to start several threads in the configuration using the
"nbthread" directive in the global section. It's possible to force to
map threads to CPUs but we're not completely satisfied with the current
configuration options and will be working on them (any feedback is
welcome). Another point is that we tried to start to take a look at
the device detection extensions and it didn't appear trivial enough
to fit in the tight schedule so this was postponed. Their maintainers
are welcome to take a look as they know this much better than anyone
else, and the thread developers will be happy to give some help on the
subject.

Important note: I just found that health-checks are totally broken
(recursive locks and missed unlocks), so please disable checks when
testing threads for now. We'll see how to address this.

  - small object cache : this is what I've been calling the "favicon cache"
for several years now. The idea is always the same, when haproxy is
deployed in front of a slow application server having to deliver a few
small static objects, it costs a lot to this server to deliver them. A
small cache of a few megabytes caching small objects for a short time
with zero administration definitely helps here. As we don't want this
cache to cause trouble, it's pessimistic : if any risk is suspected,
an object is not cached ; and by default it caches for a short time
(1 minute I believe) so that even after a failed deployment, it takes
less time to wait than to wake up the LB admin to clear the cache. To
be very clear, this is not meant to replace any real cache you might
already be using. Maybe it could offload it from some dumb files at
best. It's only meant to improve the situation for those not having
a cache. The cache is enabled using "http-request cache-use" and
"http-response cache-store" directives like below (the doc will
come soon) :

listen frt
mode http
http-response cache-store foobar
http-request cache-use foobar

cache foobar
total-max-size 4   # size in megabytes

  - client-facing HTTP/2 : that's HTTP/2 support on the frontend. It's
much better after the completely new rewrite than the first attempt a
few months ago, and in the end I'm really happy with the outcome
despite the pain it was. It's now almost complete, it supports POST,
1xx responses, chunked responses. In fact it's easier to enumerate
what it does not support : CONTINUATION frames are not yet implemented
(I may have found how to do it), PRIORITY frames are ignored (that's
allowed by the spec), there's no equivalent of chunked encoding in
requests, and trailers from the response are discarded. What it needs
most is real world exposure now to spot corner cases. I wanted to
place it on haproxy.org until I failed on the thread problem described
above and had to revert. In order to enable it, simply add
"alpn http/1.1,h2" on a "bind" line present in an HTTP mode frontend,
and if

[PATCH] BUG/MINOR: send-proxy-v2

2017-10-31 Thread Emmanuel Hocdet 
Hi Willy,

I find 2 (old) bugs in send-proxy-v2.
Can you consider this patches?

++
Manu




0001-BUG-MINOR-send-proxy-v2-fix-dest_len-in-make_tlv-cal.patch
Description: Binary data


0002-BUG-MINOR-send-proxy-v2-string-size-must-include-0.patch
Description: Binary data

[PATCH] BUG/MEDIUM: threads: Initialize the sync-point

2017-10-31 Thread Christopher Faulet 

Hi,

A line was lost in one of our merges about threads. Sorry...

--
Christopher
>From 72146f0b689f952df46fe576f74a3bd7a3bf31e6 Mon Sep 17 00:00:00 2001
From: Christopher Faulet 
Date: Tue, 31 Oct 2017 17:30:12 +0100
Subject: [PATCH] BUG/MEDIUM: threads: Initialize the sync-point

The sync point must be initialized before starting threads. This line was lost
in one of merges preparing the threads support integration.
---
 src/haproxy.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/src/haproxy.c b/src/haproxy.c
index 2d2d87723..ed6c8ba8d 100644
--- a/src/haproxy.c
+++ b/src/haproxy.c
@@ -2806,6 +2806,8 @@ int main(int argc, char **argv)
 		pthread_t*threads = calloc(global.nbthread, sizeof(pthread_t));
 		int  i;
 
+		THREAD_SYNC_INIT((1UL << global.nbthread) - 1);
+
 		/* Init tids array */
 		for (i = 0; i < global.nbthread; i++)
 			tids[i] = i;
-- 
2.13.6

Re: [PATCHES][ssl] Add 0-RTT support with OpenSSL 1.1.1

2017-10-31 Thread Willy Tarreau 
On Tue, Oct 31, 2017 at 01:35:16PM +0100, Olivier Houchard wrote:
> The attached patch removes the global ssl-allow-0rtt option.

Merged, thanks!
Willy

Re: [PATCH] Fix SRV records again

2017-10-31 Thread Willy Tarreau 
On Tue, Oct 31, 2017 at 03:27:02PM +0100, Olivier Houchard wrote:
> The recently merged MT code broke SRV records by attempting to recursively
> lock the DNS lock. The attached patch attemps to fix this by letting the
> relevant code know if the lock is already held or not.

Merged, thanks Olivier.
Willy

[PATCH] Fix SRV records again

2017-10-31 Thread Olivier Houchard 
Hi,

The recently merged MT code broke SRV records by attempting to recursively
lock the DNS lock. The attached patch attemps to fix this by letting the
relevant code know if the lock is already held or not.

Regards,

Olivier
>From 4df4df3bc00ddf9a1e7b94188058a0d76816 Mon Sep 17 00:00:00 2001
From: Olivier Houchard 
Date: Tue, 31 Oct 2017 15:21:19 +0100
Subject: [PATCH] BUG/MINOR: dns: Fix SRV records with the new thread code.

srv_set_fqdn() may be called with the DNS lock already held, but tries to
lock it anyway. So, add a new parameter to let it know if it was already
locked or not;
---
 include/proto/server.h |  2 +-
 src/dns.c  |  2 +-
 src/server.c   | 21 -
 3 files changed, 14 insertions(+), 11 deletions(-)

diff --git a/include/proto/server.h b/include/proto/server.h
index ff4ec77ca..14f4926e2 100644
--- a/include/proto/server.h
+++ b/include/proto/server.h
@@ -56,7 +56,7 @@ extern struct list updated_servers;
 
 /* functions related to server name resolution */
 int snr_update_srv_status(struct server *s, int has_no_ip);
-const char *update_server_fqdn(struct server *server, const char *fqdn, const 
char *updater);
+const char *update_server_fqdn(struct server *server, const char *fqdn, const 
char *updater, int dns_locked);
 int snr_resolution_cb(struct dns_requester *requester, struct dns_nameserver 
*nameserver);
 int snr_resolution_error_cb(struct dns_requester *requester, int error_code);
 struct server *snr_check_ip_callback(struct server *srv, void *ip, unsigned 
char *ip_family);
diff --git a/src/dns.c b/src/dns.c
index 1fdc461e6..a8d468cf4 100644
--- a/src/dns.c
+++ b/src/dns.c
@@ -545,7 +545,7 @@ static void dns_check_dns_response(struct dns_resolution 
*res)
if (dns_dn_label_to_str(item->target, 
item->data_len+1,
hostname, 
DNS_MAX_NAME_SIZE) == -1)
continue;
-   msg = update_server_fqdn(srv, hostname, "SRV 
record");
+   msg = update_server_fqdn(srv, hostname, "SRV 
record", 1);
if (msg)
send_log(srv->proxy, LOG_NOTICE, "%s", 
msg);
 
diff --git a/src/server.c b/src/server.c
index 2d0e3b4f9..37f90d8c9 100644
--- a/src/server.c
+++ b/src/server.c
@@ -53,7 +53,7 @@ HA_SPINLOCK_T updated_servers_lock;
 static void srv_register_update(struct server *srv);
 static void srv_update_state(struct server *srv, int version, char **params);
 static int srv_apply_lastaddr(struct server *srv, int *err_code);
-static int srv_set_fqdn(struct server *srv, const char *fqdn);
+static int srv_set_fqdn(struct server *srv, const char *fqdn, int dns_locked);
 
 /* List head of all known server keywords */
 static struct srv_kw_list srv_keywords = {
@@ -2911,7 +2911,7 @@ static void srv_update_state(struct server *srv, int 
version, char **params)
 * from stats socket).
 */
if (fqdn_set_by_cli) {
-   srv_set_fqdn(srv, fqdn);
+   srv_set_fqdn(srv, fqdn, 0);
srv->next_admin |= 
SRV_ADMF_HMAINT;
}
}
@@ -3764,13 +3764,14 @@ int srv_set_addr_via_libc(struct server *srv, int 
*err_code)
 /* Set the server's FDQN (->hostname) from .
  * Returns -1 if failed, 0 if not.
  */
-int srv_set_fqdn(struct server *srv, const char *hostname)
+int srv_set_fqdn(struct server *srv, const char *hostname, int dns_locked)
 {
struct dns_resolution *resolution;
char  *hostname_dn;
inthostname_len, hostname_dn_len;
 
-   SPIN_LOCK(DNS_LOCK, >resolvers->lock);
+   if (!dns_locked)
+   SPIN_LOCK(DNS_LOCK, >resolvers->lock);
/* run time DNS resolution was not active for this server
 * and we can't enable it at run time for now.
 */
@@ -3805,11 +3806,13 @@ int srv_set_fqdn(struct server *srv, const char 
*hostname)
goto err;
 
   end:
-   SPIN_UNLOCK(DNS_LOCK, >resolvers->lock);
+   if (!dns_locked)
+   SPIN_UNLOCK(DNS_LOCK, >resolvers->lock);
return 0;
 
   err:
-   SPIN_UNLOCK(DNS_LOCK, >resolvers->lock);
+   if (!dns_locked)
+   SPIN_UNLOCK(DNS_LOCK, >resolvers->lock);
return -1;
 }
 
@@ -3936,7 +3939,7 @@ int srv_init_addr(void)
return return_code;
 }
 
-const char *update_server_fqdn(struct server *server, const char *fqdn, const 
char *updater)
+const char *update_server_fqdn(struct server *server, const char *fqdn, const 
char *updater, int dns_locked)
 {
 
struct chunk *msg;
@@ -3957,7 +3960,7 @@

Re: confusion regarding usage of haproxy for large number of connections

2017-10-31 Thread Lukas Tribus 
Hello,


2017-10-31 6:55 GMT+01:00 Baptiste :
> hi
>
> You miss a "maxconn 8000" in your frontend as well.
> maxconn in the global section is process-wide, but it does not apply to the
> frontend (which is limited to 2000 connections by default).

However "maxconn 8000" is the in the default section, not the global section,
so the frontend is supposed to inherit that.



> Hi,
> I am confused regarding the readme text ' This is a development version,
> so it is expected to break from time to time,
> to add and remove features without prior notification and it should not be
> used
> in production'

Well, where do you have this release from? What release is it exactly and
what's the output of haproxy -vv? Chances are you are running a development
version, which you should absolutely avoid.

If you compile yourself, get an uptodate tarball from haproxy.org.


cheers,
lukas

Re: [PATCHES][ssl] Add 0-RTT support with OpenSSL 1.1.1

2017-10-31 Thread Olivier Houchard 
On Fri, Oct 27, 2017 at 03:54:27PM +0200, Emmanuel Hocdet wrote:
> 
> > Le 27 oct. 2017 à 15:02, Olivier Houchard  a écrit :
> > 
> > The attached patch does use the ssl_conf, instead of abusing ssl_options.
> > I also added a new field in global_ssl, I wasn't so sure about this, but
> > decided people may want to enable 0RTT globally.
> > 
> > Emmanuel, is this ok for you ?
> > 
> 
> In global option seem a bad idea.
> 
> My opinion about global ssl ‘options’ for bind.
> . Good fit is in ssl-default-bind-options. It can be extend to more options 
> like
> generate-cert, strict-sni, ….
> (In this case have a kw_list will be good idea to have something better than 
> parsing in if/then/else
> in ssl_parse_default_bind_options)
> . Some options have already 2 locations for configuration (bind line and per 
> certificats), we really
> need a third? And some options are not really good candidate.
> 
> ++
> Manu
> 

Hi,

The attached patch removes the global ssl-allow-0rtt option.

Regards,

Olivier
>From 119a9c1b5324c4ef0636bc35d8e431a17c287076 Mon Sep 17 00:00:00 2001
From: Olivier Houchard 
Date: Tue, 31 Oct 2017 13:32:10 +0100
Subject: [PATCH] MINOR: ssl: Remove the global allow-0rtt option.

---
 doc/configuration.txt |  4 
 src/ssl_sock.c| 20 
 2 files changed, 24 deletions(-)

diff --git a/doc/configuration.txt b/doc/configuration.txt
index 8d0624839..67b888905 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -847,10 +847,6 @@ resetenv [ ...]
   next line in the configuration file sees the new environment. See also
   "setenv", "presetenv", and "unsetenv".
 
-ssl-allow-0rtt
-  Allow using 0RTT on every listener. 0RTT is prone to various attacks, so be
-  sure to know the security implications before activating it.
-
 stats bind-process [ all | odd | even | [-] ] ...
   Limits the stats socket to a certain set of processes numbers. By default the
   stats socket is bound to all processes, causing a warning to be emitted when
diff --git a/src/ssl_sock.c b/src/ssl_sock.c
index 13d952652..7f52c4057 100644
--- a/src/ssl_sock.c
+++ b/src/ssl_sock.c
@@ -166,7 +166,6 @@ static struct {
char *crt_base; /* base directory path for certificates */
char *ca_base;  /* base directory path for CAs and CRLs */
int  async; /* whether we use ssl async mode */
-   int default_early_data; /* Shall we default to allow early data */
 
char *listen_default_ciphers;
char *connect_default_ciphers;
@@ -7325,7 +7324,6 @@ static int bind_parse_ssl(char **args, int cur_arg, 
struct proxy *px, struct bin
conf->ssl_conf.ciphers = 
strdup(global_ssl.listen_default_ciphers);
conf->ssl_options |= global_ssl.listen_default_ssloptions;
conf->ssl_conf.ssl_methods.flags |= 
global_ssl.listen_default_sslmethods.flags;
-   conf->ssl_conf.early_data = global_ssl.default_early_data;
if (!conf->ssl_conf.ssl_methods.min)
conf->ssl_conf.ssl_methods.min = 
global_ssl.listen_default_sslmethods.min;
if (!conf->ssl_conf.ssl_methods.max)
@@ -7819,23 +7817,6 @@ static int ssl_parse_global_ca_crt_base(char **args, int 
section_type, struct pr
return 0;
 }
 
-/* parse the "ssl-allow-0rtt" keyword in global section.
- * Returns <0 on alert, >0 on warning, 0 on success.
- */
-static int ssl_parse_global_ssl_allow_0rtt(char **args, int section_type,
-struct proxy *curpx, struct proxy *defpx, const char *file, int line,
-char **err)
-{
-#if (OPENSSL_VERSION_NUMBER >= 0x10101000L)
-global_ssl.default_early_data = 1;
-return 0;
-#else
-memprintf(err, "'%s': openssl library does not early data", args[0]);
-return -1;
-#endif
-
-}
-
 /* parse the "ssl-mode-async" keyword in global section.
  * Returns <0 on alert, >0 on warning, 0 on success.
  */
@@ -8526,7 +8507,6 @@ static struct cfg_kw_list cfg_kws = {ILH, {
{ CFG_GLOBAL, "ca-base",  ssl_parse_global_ca_crt_base },
{ CFG_GLOBAL, "crt-base", ssl_parse_global_ca_crt_base },
{ CFG_GLOBAL, "maxsslconn", ssl_parse_global_int },
-   { CFG_GLOBAL, "ssl-allow-0rtt", ssl_parse_global_ssl_allow_0rtt },
{ CFG_GLOBAL, "ssl-default-bind-options", 
ssl_parse_default_bind_options },
{ CFG_GLOBAL, "ssl-default-server-options", 
ssl_parse_default_server_options },
 #ifndef OPENSSL_NO_DH
-- 
2.13.5