Re: Issues with redirects for POST requests with payload

2017-05-08 Thread Ciprian Dorin Craciun 
On Sat, May 6, 2017 at 11:13 AM, Ciprian Dorin Craciun
 wrote:
> Hello all!
>
> In last weeks I've started encountering a problem, that for our
> particular use-case is seriously breaking some of our sites, namely:
>
> * a client makes a POST request which has a "largish" payload, one
> that does not manage to "push" it through before HAProxy has a chance
> to respond,
> * if HAProxy is configured to redirect such a request (like for
> example upgrading HTTP to HTTPS),
> * then HAProxy responds with the redirect, and closes the connection;
> but the client has not yet been able to push its POST body and
> receives a write error, and thus it aborts without trying to read the
> response from HAProxy;
>
>
> One can easily reproduce this with:
> (
>   printf -- 'POST /invalid HTTP/1.1\r\nHost: invalid.example.com\r\n\r\n'
>   dd if=/dev/urandom bs=1024 count=4 | base64
> ) \
> | socat -d -d -v tcp:127.0.0.1:80,sndbuf=16 stdio
>
> , which results in a connection reset, as `socat` is trying to push
> data to a closed socket.
>
> (Via private email I can give an actual `tcpdump` capture with production 
> data.)
>
>
>
>
> Unfortunately this issue doesn't impact a "random" client but recent
> versions of Firefox and Chrome, which just display a "connection
> reset" kind of message to the users.
>
>
> I've tried searching for a similar problem, and found this:
>
>   
> http://haproxy.formilux.narkive.com/9xhXJk4f/redirecting-on-a-large-post-without-reading-it-entirely
>   http://haproxy.formilux.narkive.com/gYztlqms/fwd-302-to-502-error
>
>
> But it's not clear to me if these issues were fixed since almost 8
> years ago, or how should I proceed in solving this issue myself.  (I'm
> open to applying patches and re-compiling HAProxy.)
>
>
> Increasing `tune.bufsize` to 128k doesn't seem to help either.
>
> (I am using HAProxy 1.6.11.)




Just wanted to "ping" this thread, as perhaps sending the original
email on weekend got it "forgotten".  :)

Or perhaps nobody has hit this issue in production (yet)?

Thanks,
Ciprian.

Re: reqrep syntax

2017-05-08 Thread Aristedes Maniatis 
On 9/5/17 8:26am, Holger Just wrote:
> With a modern HAProxy (i.e. anything >= 1.6), you can just use
> 
> http-request set-path %[path,regsub(^/static,/)]

Brilliant, thanks. I spent 20 minutes trying to figure how to submit a patch 
for the documentation, but failed completely. Maintainers of this project: 
please consider making it easier to submit documentation fixes. I know it can 
involve more work engaging with a community, but the rewards can be useful. 
With a task tracker, pull requests, etc its a little hard for a casual user to 
help.

The things that need improvement in the docs are:

1. In the documentation for reqrep make a big note that the examples provided 
are not the current recommended approach and to instead use http-request if 
possible. Otherwise the official documentation reads like it is best practice.

2. In the docs for reqrep, put in an example of what the regex is matching. For 
example "GET /some/path?param=true HTTP/1.1"

3. In the docs for http-request [1] separate out the possible parameters on new 
lines. I read that doc 10 times and never noticed I needed to read across 
because someone wanted to save paper when the docs were printed by wrapping the 
parameters. So I never saw set-path.

4. In the cbonte html version, it would be lovely if set-path was a searchable 
keyword. "path" is and that's just an acl parameter.


Thanks for your help and to everyone working on this project.

Cheers
Ari


[1] https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#http-request


-- 
-->
Aristedes Maniatis
CEO, ish
https://www.ish.com.au
GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A



signature.asc
Description: OpenPGP digital signature

Re: haproxy not creating stick-table entries fast enough

2017-05-08 Thread redundantl y 
On Mon, May 8, 2017 at 4:01 PM, Lukas Tribus  wrote:

> Hello,
>
>
> Am 09.05.2017 um 00:38 schrieb redundantl y:
> > I am running haproxy 1.5.18-3 on CentOS 7 and need to use the
> > stick-table feature to make sure traffic for a specific user persists
> > to a given server.
> >
> > Things work fine when connections come in slowly, however when there's
> > numerous simultaneous connections and a stick-table entry doesn't
> > exist yet some requests will be sent to both backend servers until
> > they eventually stick to just one.
> >
> > For example, using Apache Bench to test:
> >
> > ab -c 10 -n 30 'http://example.com/index.php?email=a...@example.com'
> >
> > I see this in the haproxy log:
> >
> > May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50812
> >  [08/May/2017:14:49:10.934] http_front
> > backend/server-1 0/0/0/7/7 200 222 - -  9/9/9/4/0 0/0 "GET
> > /index.php?email=a...@example.com  HTTP/1.0"
> > May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50811
> >  [08/May/2017:14:49:10.933] http_front
> > backend/server-2 0/0/0/8/8 200 222 - -  8/8/8/4/0 0/0 "GET
> > /index.php?email=a...@example.com  HTTP/1.0"
> > May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50816
> >  [08/May/2017:14:49:10.935] http_front
> > backend/server-1 0/0/0/7/7 200 222 - -  7/7/7/1/0 0/0 "GET
> > /index.php?email=a...@example.com  HTTP/1.0"
> > May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50819
> >  [08/May/2017:14:49:10.935] http_front
> > backend/server-2 0/0/1/6/7 200 222 - -  6/6/6/1/0 0/0 "GET
> > /index.php?email=a...@example.com  HTTP/1.0"
> > May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50814
> >  [08/May/2017:14:49:10.935] http_front
> > backend/server-1 0/0/0/7/7 200 222 - -  5/5/5/1/0 0/0 "GET
> > /index.php?email=a...@example.com  HTTP/1.0"
> > May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50810
> >  [08/May/2017:14:49:10.933] http_front
> > backend/server-1 0/0/0/9/9 200 222 - -  4/4/4/0/0 0/0 "GET
> > /index.php?email=a...@example.com  HTTP/1.0"
> > May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50813
> >  [08/May/2017:14:49:10.934] http_front
> > backend/server-2 0/0/0/8/8 200 222 - -  3/3/3/0/0 0/0 "GET
> > /index.php?email=a...@example.com  HTTP/1.0"
> > May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50815
> >  [08/May/2017:14:49:10.935] http_front
> > backend/server-2 0/0/0/7/7 200 222 - -  2/2/2/0/0 0/0 "GET
> > /index.php?email=a...@example.com  HTTP/1.0"
> > May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50817
> >  [08/May/2017:14:49:10.935] http_front
> > backend/server-2 0/0/0/7/8 200 222 - -  1/1/1/0/0 0/0 "GET
> > /index.php?email=a...@example.com  HTTP/1.0"
> > May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50818
> >  [08/May/2017:14:49:10.935] http_front
> > backend/server-1 0/0/1/6/8 200 222 - -  0/0/0/0/0 0/0 "GET
> > /index.php?email=a...@example.com  HTTP/1.0"
> > May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50820
> >  [08/May/2017:14:49:10.967] http_front
> > backend/server-1 0/0/0/5/5 200 222 - -  3/3/2/2/0 0/0 "GET
> > /index.php?email=a...@example.com  HTTP/1.0"
> > May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50821
> >  [08/May/2017:14:49:10.968] http_front
> > backend/server-1 0/0/0/4/4 200 222 - -  2/2/2/2/0 0/0 "GET
> > /index.php?email=a...@example.com  HTTP/1.0"
> > May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50823
> >  [08/May/2017:14:49:10.972] http_front
> > backend/server-1 0/0/1/5/6 200 222 - -  7/7/7/7/0 0/0 "GET
> > /index.php?email=a...@example.com  HTTP/1.0"
> > May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50822
> >  [08/May/2017:14:49:10.972] http_front
> > backend/server-1 0/0/0/8/8 200 222 - -  6/6/6/6/0 0/0 "GET
> > /index.php?email=a...@example.com  HTTP/1.0"
> > [...]
> >
> > After this point haproxy correctly sends all traffic to server-1. When
> > the stick-table entry expires the problem occurs again.
> >
> > I have tried persisting off a url parameter and source address, both
> > exhibit the same issue.
> >
> > Is haproxy unable to properly handle numerous simultaneous
> > (concurrent) requests like this? Is there something I can do to get
> > this to work as desired?
>
> Those 10 concurrent requests are not elaborated sequentially, haproxy is
> a event-loop based application and handles those request in

Re: haproxy not creating stick-table entries fast enough

2017-05-08 Thread Lukas Tribus 
Hello,


Am 09.05.2017 um 00:38 schrieb redundantl y:
> I am running haproxy 1.5.18-3 on CentOS 7 and need to use the
> stick-table feature to make sure traffic for a specific user persists
> to a given server.
>
> Things work fine when connections come in slowly, however when there's
> numerous simultaneous connections and a stick-table entry doesn't
> exist yet some requests will be sent to both backend servers until
> they eventually stick to just one.
>
> For example, using Apache Bench to test:
>
> ab -c 10 -n 30 'http://example.com/index.php?email=a...@example.com'
>
> I see this in the haproxy log:
>
> May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50812
>  [08/May/2017:14:49:10.934] http_front
> backend/server-1 0/0/0/7/7 200 222 - -  9/9/9/4/0 0/0 "GET
> /index.php?email=a...@example.com  HTTP/1.0"
> May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50811
>  [08/May/2017:14:49:10.933] http_front
> backend/server-2 0/0/0/8/8 200 222 - -  8/8/8/4/0 0/0 "GET
> /index.php?email=a...@example.com  HTTP/1.0"
> May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50816
>  [08/May/2017:14:49:10.935] http_front
> backend/server-1 0/0/0/7/7 200 222 - -  7/7/7/1/0 0/0 "GET
> /index.php?email=a...@example.com  HTTP/1.0"
> May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50819
>  [08/May/2017:14:49:10.935] http_front
> backend/server-2 0/0/1/6/7 200 222 - -  6/6/6/1/0 0/0 "GET
> /index.php?email=a...@example.com  HTTP/1.0"
> May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50814
>  [08/May/2017:14:49:10.935] http_front
> backend/server-1 0/0/0/7/7 200 222 - -  5/5/5/1/0 0/0 "GET
> /index.php?email=a...@example.com  HTTP/1.0"
> May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50810
>  [08/May/2017:14:49:10.933] http_front
> backend/server-1 0/0/0/9/9 200 222 - -  4/4/4/0/0 0/0 "GET
> /index.php?email=a...@example.com  HTTP/1.0"
> May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50813
>  [08/May/2017:14:49:10.934] http_front
> backend/server-2 0/0/0/8/8 200 222 - -  3/3/3/0/0 0/0 "GET
> /index.php?email=a...@example.com  HTTP/1.0"
> May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50815
>  [08/May/2017:14:49:10.935] http_front
> backend/server-2 0/0/0/7/7 200 222 - -  2/2/2/0/0 0/0 "GET
> /index.php?email=a...@example.com  HTTP/1.0"
> May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50817
>  [08/May/2017:14:49:10.935] http_front
> backend/server-2 0/0/0/7/8 200 222 - -  1/1/1/0/0 0/0 "GET
> /index.php?email=a...@example.com  HTTP/1.0"
> May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50818
>  [08/May/2017:14:49:10.935] http_front
> backend/server-1 0/0/1/6/8 200 222 - -  0/0/0/0/0 0/0 "GET
> /index.php?email=a...@example.com  HTTP/1.0"
> May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50820
>  [08/May/2017:14:49:10.967] http_front
> backend/server-1 0/0/0/5/5 200 222 - -  3/3/2/2/0 0/0 "GET
> /index.php?email=a...@example.com  HTTP/1.0"
> May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50821
>  [08/May/2017:14:49:10.968] http_front
> backend/server-1 0/0/0/4/4 200 222 - -  2/2/2/2/0 0/0 "GET
> /index.php?email=a...@example.com  HTTP/1.0"
> May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50823
>  [08/May/2017:14:49:10.972] http_front
> backend/server-1 0/0/1/5/6 200 222 - -  7/7/7/7/0 0/0 "GET
> /index.php?email=a...@example.com  HTTP/1.0"
> May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50822
>  [08/May/2017:14:49:10.972] http_front
> backend/server-1 0/0/0/8/8 200 222 - -  6/6/6/6/0 0/0 "GET
> /index.php?email=a...@example.com  HTTP/1.0"
> [...]
>
> After this point haproxy correctly sends all traffic to server-1. When
> the stick-table entry expires the problem occurs again.
>
> I have tried persisting off a url parameter and source address, both
> exhibit the same issue.
>
> Is haproxy unable to properly handle numerous simultaneous
> (concurrent) requests like this? Is there something I can do to get
> this to work as desired?

Those 10 concurrent requests are not elaborated sequentially, haproxy is
a event-loop based application and handles those request in parallel.
Meaning in this case all of those 10 request are load-balanced without
stickiness.

Use "-c 1" and maybe "-k" to enable keepalive in ab, which will be more
inline with what happens in the real world.


Regards,
Lukas

haproxy not creating stick-table entries fast enough

2017-05-08 Thread redundantl y 
I am running haproxy 1.5.18-3 on CentOS 7 and need to use the stick-table
feature to make sure traffic for a specific user persists to a given server.

Things work fine when connections come in slowly, however when there's
numerous simultaneous connections and a stick-table entry doesn't exist yet
some requests will be sent to both backend servers until they eventually
stick to just one.

For example, using Apache Bench to test:

ab -c 10 -n 30 'http://example.com/index.php?email=a...@example.com'

I see this in the haproxy log:

May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50812
[08/May/2017:14:49:10.934] http_front backend/server-1 0/0/0/7/7 200 222 -
-  9/9/9/4/0 0/0 "GET /index.php?email=a...@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50811
[08/May/2017:14:49:10.933] http_front backend/server-2 0/0/0/8/8 200 222 -
-  8/8/8/4/0 0/0 "GET /index.php?email=a...@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50816
[08/May/2017:14:49:10.935] http_front backend/server-1 0/0/0/7/7 200 222 -
-  7/7/7/1/0 0/0 "GET /index.php?email=a...@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50819
[08/May/2017:14:49:10.935] http_front backend/server-2 0/0/1/6/7 200 222 -
-  6/6/6/1/0 0/0 "GET /index.php?email=a...@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50814
[08/May/2017:14:49:10.935] http_front backend/server-1 0/0/0/7/7 200 222 -
-  5/5/5/1/0 0/0 "GET /index.php?email=a...@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50810
[08/May/2017:14:49:10.933] http_front backend/server-1 0/0/0/9/9 200 222 -
-  4/4/4/0/0 0/0 "GET /index.php?email=a...@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50813
[08/May/2017:14:49:10.934] http_front backend/server-2 0/0/0/8/8 200 222 -
-  3/3/3/0/0 0/0 "GET /index.php?email=a...@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50815
[08/May/2017:14:49:10.935] http_front backend/server-2 0/0/0/7/7 200 222 -
-  2/2/2/0/0 0/0 "GET /index.php?email=a...@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50817
[08/May/2017:14:49:10.935] http_front backend/server-2 0/0/0/7/8 200 222 -
-  1/1/1/0/0 0/0 "GET /index.php?email=a...@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50818
[08/May/2017:14:49:10.935] http_front backend/server-1 0/0/1/6/8 200 222 -
-  0/0/0/0/0 0/0 "GET /index.php?email=a...@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50820
[08/May/2017:14:49:10.967] http_front backend/server-1 0/0/0/5/5 200 222 -
-  3/3/2/2/0 0/0 "GET /index.php?email=a...@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50821
[08/May/2017:14:49:10.968] http_front backend/server-1 0/0/0/4/4 200 222 -
-  2/2/2/2/0 0/0 "GET /index.php?email=a...@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50823
[08/May/2017:14:49:10.972] http_front backend/server-1 0/0/1/5/6 200 222 -
-  7/7/7/7/0 0/0 "GET /index.php?email=a...@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50822
[08/May/2017:14:49:10.972] http_front backend/server-1 0/0/0/8/8 200 222 -
-  6/6/6/6/0 0/0 "GET /index.php?email=a...@example.com HTTP/1.0"
[...]

After this point haproxy correctly sends all traffic to server-1. When the
stick-table entry expires the problem occurs again.

I have tried persisting off a url parameter and source address, both
exhibit the same issue.

Is haproxy unable to properly handle numerous simultaneous (concurrent)
requests like this? Is there something I can do to get this to work as
desired?

Thanks.

Re: Automatic Certificate Switching Idea

2017-05-08 Thread Lukas Tribus 
Hello,


Am 30.04.2017 um 22:16 schrieb Daniel Schneller:
> Hi!
>
> Yes, you got it right. I have no idea if there are technical limitations in 
> the SSL library or other parts of the code that would make several 
> certificate/key pairs for the same domain infeasible. 
>
> If there were hard restrictions, it could certainly be done "externally" with 
> a set of clever scripts and haproxy reloads, but IMO it would be a less 
> brittle solution if it were built right in. Also, it wouldn't require 
> separate scripts per platform and init system (SysV, systemd, …). 
>
> Daniel

Similar to the "Certificate order" thread, this is a very specific
requirement that only a handful of people require and it is imho best
achieved with external configuration tools. If we implement everyone of
those use-cases we are gonna transform haproxy into Microsoft Office.

I'm opposed to heavy feature-bloating for provisioning use-cases, that
can quite easily fixed where the fix belongs - the provisioning layer.


Those are just my two cents though ...


cheers,
lukas

Re: reqrep syntax

2017-05-08 Thread Holger Just 
Hi Ari,

Aristedes Maniatis wrote:
> In the manual [1] there is an example for using reqrep with syntax
> like this:
> 
> # replace "/static/" with "/" at the beginning of any request path. 
> reqrep ^([^\ :]*)\ /static/(.*) \1\ /\2
> 
> [...]
> 
> Firstly, is there no better/cleaner way to rewrite the path than this
> messy regex which seems to process all the http headers rather than
> just the request line that contains the path?

With a modern HAProxy (i.e. anything >= 1.6), you can just use

http-request set-path %[path,regsub(^/static,/)]

Generally, you should always use the http-request rules instead of the
(older) reg* rules since the http-request rules are much more powerful,
cleaner and have the advantage of always being evaluated in the defined
order which the reg* rules are not.

Old guides like the one you linked to in [2] were written before the
http-request rules got as powerful as they are today. The guide was
written for HAProxy 1.4 which is quite outdated now.

> Secondly, why was this particular regex used in the example? If I'm
> reading this correctly, quoting the string would simplify it to:
> 
> reqrep '^([^ :]*) /static/(.*)' '\1 /\2'

String quoting was only introduced in HAProxy 1.6. Before that (and
during the time reqrep was the only way to manipulate HTTP requests),
you always had to escape spaces with a backslash.

> But is this even clearer and safer:
> 
> reqrep '^([A-Z]+) /static(/.*)' '\1 \2'
> 
> or narrow this down (where appropriate to the type of request) to
> 
> reqrep '^(GET|POST) /static(/.*)' '\1 \2'
> 
> Perhaps the example would be clearer like this (even though this is
> slower and not needed here, it is clearly demonstrating the point):
> 
> reqrep '^(GET|POST) /static(/.*) (HTTP/.+)' '\1 \2 \3'

These are all valid variations for the common case. Still, users might
decide to also send lower-case HTTP verbs or other variations. The
syntax from the guides is the most generic one.

Still, you should probably just get rid of any regrep rules and just use
http-request.

Cheers,
Holger

[2]
https://fromanegg.com/post/2014/12/05/how-to-rewrite-and-redirect-with-haproxy/

Re: Passing SNI value ( ssl_fc_sni ) to backend's verifyhost.

2017-05-08 Thread Lukas Tribus 
Hello,


Am 08.05.2017 um 10:56 schrieb Daniel Schneller:
> Just my 2c, I very much support Kevin’s argument.
> Even though we are not (yet) verifying backends — because currently we
> _are_ in a private LAN — we are planning to deploy parts of our
> application to public cloud infrastructure soon, so it would be a
> quite important feature.
>
>> On 6. May. 2017, at 19:18, Kevin McArthur > > wrote:
>>
>> 1. The Snowden leaks and the whole "SSL added and removed here"
>> issue, for example. TLS on internal networks is more important these
>> days due to local network implants and other security issues on LANs.
>>
>> 2. Our use case is actually DigitalOcean where there is "private
>> networking" but it is shared among many customers. Operating without
>> TLS on this semi-private network would be unwise.
>>
>> 3. Most of the public tutorials for re-encrypt bridged TLS are simply
>> incurring TLS overhead while providing no TLS security. (eg SSL on
>> but, verify none enabled, verifyhost not set, etc)
>>
>> 4. Use cases like CDN proxy of public servers. Think Cloudflare's
>> Full SSL (Strict) setup...
>>
>>


Haproxy can verify the certificate of backend TLS servers since day 1.

The only thing missing is client SNI based backend certificate
verification, which yes - since we can pass client SNI to the TLS server
- we need to consider for the certificate verification process as well.


Regards,
Lukas

reqrep syntax

2017-05-08 Thread Aristedes Maniatis 
In the manual [1] there is an example for using reqrep with syntax like this:

# replace "/static/" with "/" at the beginning of any request path.
reqrep ^([^\ :]*)\ /static/(.*) \1\ /\2


But a typical http request looks like this:

GET /haproxy-dconv/1.7/configuration.html HTTP/1.1
Host: cbonte.github.io
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:53.0) 
Gecko/20100101 Firefox/53.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate, br
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0


Firstly, is there no better/cleaner way to rewrite the path than this messy 
regex which seems to process all the http headers rather than just the request 
line that contains the path? And not only that, but we need to consume all 
three parts of that line including the HTTP version.

The ACL processing engine already can get at the path, so it is available 
somewhere inside haproxy, just not in a way we can easily rewrite.


Secondly, why was this particular regex used in the example? If I'm reading 
this correctly, quoting the string would simplify it to:

reqrep '^([^ :]*) /static/(.*)' '\1 /\2'

But is this even clearer and safer:

reqrep '^([A-Z]+) /static(/.*)' '\1 \2'

or narrow this down (where appropriate to the type of request) to

reqrep '^(GET|POST) /static(/.*)' '\1 \2'


Perhaps the example would be clearer like this (even though this is slower and 
not needed here, it is clearly demonstrating the point):

reqrep '^(GET|POST) /static(/.*) (HTTP/.+)' '\1 \2 \3'



Have I got all the above right? I'm finding examples on the internet like this 
[2] where the author believes the first part is the url scheme which I think is 
wrong.



Thanks
Ari





[1] https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#4.2-reqrep
[2] 
https://fromanegg.com/post/2014/12/05/how-to-rewrite-and-redirect-with-haproxy/

-- 
-->
Aristedes Maniatis
CEO, ish
https://www.ish.com.au
GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A




signature.asc
Description: OpenPGP digital signature

Re: [RFC][PATCHES] seamless reload

2017-05-08 Thread Olivier Houchard 
Hi Pavlos,

On Sun, May 07, 2017 at 12:05:28AM +0200, Pavlos Parissis wrote:
[...]
> Ignore ignore what I wrote, I am an idiot I am an idiot as I forgot the most
> important bit of the test, to enable the seamless reload by suppling the
> HAPROXY_STATS_SOCKET environment variable:-(
> 
> I added to the systemd overwrite file:
> [Service]
> Environment=CONFIG="/etc/lb_engine/haproxy.cfg"
> "HAPROXY_STATS_SOCKET=/run/lb_engine/process-1.sock"
> 
> and wrk2 reports ZERO errors where with HAPEE reports ~49.
> 
> I am terrible sorry for this stupid mistake.
> 
> But, this mistake revealed something interesting. The fact that with the 
> latest
> code we have more errors during reload.
> 
> @Olivier, great work dude. I am waiting for this to be back-ported to 
> HAPEE-1.7r1.
> 
> Once again I am sorry for my mistake,
> Pavlos
> 

Thanks a lot for testing !
This is interesting indeed. My patch may make it worse when not passing
fds via the unix socket, as all processes now keep all sockets opened, even
the one they're not using, maybe it make the window between the last
accept and the close bigger.
If that is so, then the global option "no-unused-socket" should provide
a comparable error rate.

Regards,

Olivier

Re: Compare against variable string in ACL

2017-05-08 Thread Tim Düsterhus 
Holger,

On 08.05.2017 11:37, Holger Just wrote:
> We use basically this configuration snippet in production for quite some
> years now and it works great.
> 

while not exactly pretty, it definitely works fine. Thank you.

Best regards
Tim Düsterhus

Re: Compare against variable string in ACL

2017-05-08 Thread Holger Just 
Hi Tim.

Tim Düsterhus wrote:
> I basically want an ACL that matches if 'hdr(host) == ssl_fc_sni' to use
> programming language terminology.

This is not directly possible right now using haproxy ACLs since they
are only ablle to to compare a dynamic value (the fetch) to a static
value. There is however a "trick" to still pull this off without having
to dive into Lua.

# We concatenate the HTTP Header and the SNI field in an internal header
# and check if the same value is repeated in that header.
http-request set-header X-CHECKSNI %[req.hdr(host)]==%[ssl_fc_sni] if {
ssl_fc_has_sni }

# This needs to be a named capture because of "reasons".
# Back-References to normal captures seem to be rejected by HAProxy
http-request deny if { ssl_fc_has_sni } ! { hdr(X-CHECKSNI) -m reg -i
^(?.+)==\1$ }

# Cleanup after us
http-request del-header X-CHECKSNI

We use basically this configuration snippet in production for quite some
years now and it works great.

Cheers,
Holger

Re: Passing SNI value ( ssl_fc_sni ) to backend's verifyhost.

2017-05-08 Thread Daniel Schneller 
Just my 2c, I very much support Kevin’s argument.
Even though we are not (yet) verifying backends — because currently we _are_ in 
a private LAN — we are planning to deploy parts of our application to public 
cloud infrastructure soon, so it would be a quite important feature.

Regards,
Daniel


-- 
Daniel Schneller
Principal Cloud Engineer
 
CenterDevice GmbH  | Hochstraße 11
   | 42697 Solingen
tel: +49 1754155711| Deutschland
daniel.schnel...@centerdevice.de   | www.centerdevice.de

Geschäftsführung: Dr. Patrick Peschlow, Dr. Lukas Pustina,
Michael Rosbach, Handelsregister-Nr.: HRB 18655,
HR-Gericht: Bonn, USt-IdNr.: DE-815299431


> On 6. May. 2017, at 19:18, Kevin McArthur  wrote:
> 
> 1. The Snowden leaks and the whole "SSL added and removed here" issue, for 
> example. TLS on internal networks is more important these days due to local 
> network implants and other security issues on LANs.
> 
> 2. Our use case is actually DigitalOcean where there is "private networking" 
> but it is shared among many customers. Operating without TLS on this 
> semi-private network would be unwise.
> 3. Most of the public tutorials for re-encrypt bridged TLS are simply 
> incurring TLS overhead while providing no TLS security. (eg SSL on but, 
> verify none enabled, verifyhost not set, etc)
> 
> 4. Use cases like CDN proxy of public servers. Think Cloudflare's Full SSL 
> (Strict) setup... 
> --
> 
> Kevin
> On 2017-05-05 7:20 PM, Igor Cicimov wrote:
>> 
>> 
>> On 6 May 2017 2:04 am, "Kevin McArthur" > > wrote:
>> When doing tls->haproxy->tls (bridged https) re-encryption with SNI, we need 
>> to verify the backend certificate against the SNI value requested by the 
>> client.
>> 
>> Something like server options:
>> 
>> server app1 app1.example.ca:443  ssl no-sslv3 
>> sni ssl_fc_sni verify required verifyhost ssl_fc_sni
>> 
>> However, the "verifyhost ssl_fc_sni" part doesn't work at current. Is there 
>> any chance I could get this support patched in?
>> 
>> Most folks seem to be either ignoring the backend server validation, setting 
>> verify none, or are stripping tls altogether leaving a pretty big security 
>> hole.
>> Care to elaborate why is this a security hole if the backend servers are in 
>> internal LAN which usually is the case when terminating ssl on the proxy?
>> 
>> --
>> 
>> Kevin McArthur
>> 
>