Re: HAProxy 1.5 possible bug

[Igor]

On Thu, Mar 6, 2014 at 3:50 PM, Willy Tarreau  wrote:
> We've been thinking about implementing a simple async resolver in
> combination with health checks to at least automatically update server
> addresses at EC2 and similar horrible environments where a reboot can
> change your server's address. A next step could be to try to use the
> same resolver for regular traffic. The thing is that doing this fast
> will require a cache otherwise it will be slow and will hammer the DNS
> servers quickly.

This is the most wanted feature :)


Bests,
-Igor

RE: possible crashes on linux with recent glibc

[Lukas Tribus]

Hi Willy,


>> Your description corresponds with my configuration (using select() with
>> glibc 2.15 on ubuntu crashing with some load).
>>
>>
>> On the terminal I see (which is what confuses a bit):
>> *** buffer overflow detected ***: ./haproxy terminated
>>
>> and the backtrace looks like this:
>> (gdb) backtrace full
>> #0 0xb76e2424 in __kernel_vsyscall ()
>> No symbol table info available.
>> #1 0xb755b1df in raise () from /lib/i386-linux-gnu/libc.so.6
>> No symbol table info available.
>> #2 0xb755e825 in abort () from /lib/i386-linux-gnu/libc.so.6
>> No symbol table info available.
>> #3 0xb759839a in ?? () from /lib/i386-linux-gnu/libc.so.6
>> No symbol table info available.
>> #4 0xb76310e5 in __fortify_fail () from /lib/i386-linux-gnu/libc.so.6
>> No symbol table info available.
>> #5 0xb762feba in __chk_fail () from /lib/i386-linux-gnu/libc.so.6
>> No symbol table info available.
>> #6 0xb763107a in __fdelt_warn () from /lib/i386-linux-gnu/libc.so.6
>> No symbol table info available.
>> #7 0x0809ad3f in _do_poll (p=0x80ce0e0, exp=-1820950388) at 
>> src/ev_select.c:65
>>
>>
>>
>> I'm quite sure its exactly this problem, but I prefer to double check with
>> you.
>
> Yes it was the exact same trace I used to get when using select() with
> too large file descriptors. I really think that this glibc change will
> break a large number of software...

Thanks. Yes, indeed.

I wonder why it doesn't crash without compiler optimization (-O0) though.


Anyway, thanks for confirming the backtrace.



Regards,

Lukas 

Re: possible crashes on linux with recent glibc

[Willy Tarreau]

Hi Lukas,

On Thu, Mar 06, 2014 at 09:54:44AM +0100, Lukas Tribus wrote:
> Hi Willy,
> 
> 
> >> Your description corresponds with my configuration (using select() with
> >> glibc 2.15 on ubuntu crashing with some load).
> >>
> >>
> >> On the terminal I see (which is what confuses a bit):
> >> *** buffer overflow detected ***: ./haproxy terminated
> >>
> >> and the backtrace looks like this:
> >> (gdb) backtrace full
> >> #0 0xb76e2424 in __kernel_vsyscall ()
> >> No symbol table info available.
> >> #1 0xb755b1df in raise () from /lib/i386-linux-gnu/libc.so.6
> >> No symbol table info available.
> >> #2 0xb755e825 in abort () from /lib/i386-linux-gnu/libc.so.6
> >> No symbol table info available.
> >> #3 0xb759839a in ?? () from /lib/i386-linux-gnu/libc.so.6
> >> No symbol table info available.
> >> #4 0xb76310e5 in __fortify_fail () from /lib/i386-linux-gnu/libc.so.6
> >> No symbol table info available.
> >> #5 0xb762feba in __chk_fail () from /lib/i386-linux-gnu/libc.so.6
> >> No symbol table info available.
> >> #6 0xb763107a in __fdelt_warn () from /lib/i386-linux-gnu/libc.so.6
> >> No symbol table info available.
> >> #7 0x0809ad3f in _do_poll (p=0x80ce0e0, exp=-1820950388) at 
> >> src/ev_select.c:65
> >>
> >>
> >>
> >> I'm quite sure its exactly this problem, but I prefer to double check with
> >> you.
> >
> > Yes it was the exact same trace I used to get when using select() with
> > too large file descriptors. I really think that this glibc change will
> > break a large number of software...
> 
> Thanks. Yes, indeed.
> 
> I wonder why it doesn't crash without compiler optimization (-O0) though.

I suspect that the FD_SET macros might be declared as functions instead
of macros and that they check the parameter before dereferencing the
array. That's just a guess.

Willy

ereq count on a FRONTEND for a group

[Dave]

Hello,
 What could be causing ereq counts to go up on the frontend stats field of
a normal listen group? I've got an issue where 502's are randomly being
generated for a site when the servers are online, and the only errors in
the stats are a constantly climbing amount of request errors on the
frontend - the servers in the group don't show it.

Thank you
Dave

Current solutions to the soft-restart-healthcheck-spread problem?

[Jonathan Matthews]

Hi all -

[ tl;dr How do you stop haproxy using failed backend servers immediately
after reload?
Haproxy devs, please consider implementing a
consider-servers-initially-DOWN option! ]

I wonder if people could outline how they're dealing with the combination
of these two haproxy behaviours:

1) On restart/reload/disabled-server-now-enabled-via-admin-interface,
haproxy considers a server to be 1 health check away from going down, but
considers it *initially* up.

2) On restart/reload, haproxy spreads out each backend's(?) initial server
health checks over the entire health check interval.

(If I'm slightly off with either of those statements, please forgive the
inaccuracy and let it slide for the purposes of this discussion; do let me
know if I'm /meaningfully/ wrong of course!)

The combination of these facts in a high traffic environment seems to imply
that an unhealthy-but-just-enabled server which is listed last in an
haproxy backend may receive requests for a longer-than-expected period of
time, resulting in a non-trivial number of requests failing.

In such an environment, where multiple load balancers are involved and can
be reloaded sequentially (such as mine!), it would be preferable to take a
pessimistic approach and /not/ expose servers to traffic until you're
positive that the backend is healthy, rather than haproxy's current
default-optimism approach.

I've been considering some methods to deal with this, but haven't got a
working config yet. It's getting somewhat convoluted and stick-table heavy,
so I thought I'd ask everyone:

Where you have decided that this is something you actually need to deal
with, *how* are you doing that? (I totally recognise that the combination
of a frequent health check interval and non-insane traffic volumes may mask
this issue, leading many -- myself included in previous jobs! -- not to
consider it a problem in the first place)

It's worth pointing out that I /believe/ this situation could be easily
solved (operationally) by a global, per-backend or per-server option which
switches on the pessimistic behaviour mentioned above. I recognise that
this may not be easy from an /implementation/ perspective, of course.
[Willy: any chance of an option to start each server as if it were down,
but being 1 check away from going up, rather than the opposite? :-)]

It's also worth pointing out that, whilst the "persist haproxy state over
soft restarts" concept that's been mentioned previously on list would solve
this for orderly restarts, it wouldn't solve it for crashes, reboots or
otherwise. I think the option I mentioned above would be one way to solve
it nicely, for multiple use cases.

[ For a *not* nice solution, I'll post a follow up when I get my
stick-table concept going. It's /nasty/. IMHO. Don't make me put it into
production! ;-) ]

Cheers,
Jonathan

balance leastconn does not honor weight?

[Dmitry Sivachenko]

Hello!

I am using haproxy-1.5.22.

In a single backend I have servers with different weight configured: 16, 24, 32 
(proportional to the number of CPU cores).
Most of the time they respond very fast.

When I use balance leastconn, I see in the stats web interface that they all 
receive approximately equal number of connections (Sessions->Total).
Shouldn't leastconn algorithm also honor weights of each backend (to pick a 
backend with minimal Connections/weight value)?

Thanks.

Environment variables in port part of bind?

[adam]

From the docs, it appears that HAProxy does not support environment variables 
in the port portion of the bind parameter.  Is this correct?  If it is correct, 
is there a reason for this choice, or is it just that no one has asked for the 
feature before?

Adam

Equivalent Weblogic and BigIP

[Jorge Severino]

Hi Guys:

I replaced one F5 BigIP for haproxy and everything works fine.

The only thing that not works for me is a weblogic cluster, if it works in
F5.

I can help put the equivalent parameters?

I test with indirect cookie, prefix, direct, mode http, etc.

Send Off the curl query outputs to real server, virtual Ip by F5 and
haproxy.


Direct Query a real server Weblogic:

[jseverino@jseverino Desktop]$ curl -v
http://wls2:7101/BrowserWeb/servlet/BrowserServlet
* About to connect() to wls2 port 7101 (#0)
*   Trying x.x.x.x... connected
* Connected to wls2 (x.x.x.x) port 7101 (#0)
> GET /BrowserWeb/servlet/BrowserServlet HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/
3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: wls2:7101
> Accept: */*
>
< HTTP/1.1 200 OK
< Cache-Control: no-cache,no-store
< Date: Thu, 06 Mar 2014 21:07:53 GMT
< Pragma: no-cache
< Content-Length: 6825
< Content-Type: text/html;charset=utf-8
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
< X-FRAME-OPTIONS: SAMEORIGIN
< Set-Cookie:
JSESSIONID=hGg6TYjJR82C0lhDFy5w9Qc50qlfDR74SKlHM3LykJj2XhtkMyHH!2087482587!1453328874;
path=/; HttpOnly
< X-Powered-By: Servlet/3.0 JSP/2.2


Query a F5 VS:

[jseverino@jseverino Desktop]$ curl -v
http://vipf5/BrowserWeb/servlet/BrowserServlet
* About to connect() to x.x.x.x port 80 (#0)
*   Trying x.x.x.x... connected
* Connected to x.x.x.x (x.x.x.x) port 80 (#0)
> GET /BrowserWeb/servlet/BrowserServlet HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/
3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: x.x.x.x
> Accept: */*
>
< HTTP/1.1 200 OK
< Cache-Control: no-cache,no-store
< Date: Thu, 06 Mar 2014 21:09:59 GMT
< Pragma: no-cache
< Content-Length: 6825
< Content-Type: text/html;charset=utf-8
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
< X-FRAME-OPTIONS: SAMEORIGIN
< Set-Cookie:
JSESSIONID=nzLTTYkHGQ4h2gnB2hJc3tH3rqngyyhmtp2Jn4bS1LLrZ1SJMjb2!2087482587!1453328874;
path=/; HttpOnly
< X-Powered-By: Servlet/3.0 JSP/2.2
< Set-Cookie: BIGipServervipf5-PRD=1992818698.48411.; path=/
< Vary: Accept-Encoding


And Query through Haproxy:

[jseverino@jseverino Desktop]$ curl -v
http://haproxy/BrowserWeb/servlet/BrowserServlet
* About to connect() to haproxy port 80 (#0)
*   Trying x.x.x.x... connected
* Connected to haproxy (x.x.x.x) port 80 (#0)
> GET /BrowserWeb/servlet/BrowserServlet HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/
3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: haproxy
> Accept: */*
>
< HTTP/1.1 200 OK
< Cache-Control: no-cache,no-store
< Date: Thu, 06 Mar 2014 21:12:28 GMT
< Pragma: no-cache
< Content-Length: 6825
< Content-Type: text/html;charset=utf-8
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
< X-FRAME-OPTIONS: SAMEORIGIN
< X-Powered-By: Servlet/3.0 JSP/2.2
< Set-Cookie: JSESSIONID=server01; path=/


Thaks for helpme.

Atte.
Jorge Severino

Re: Equivalent Weblogic and BigIP

[PiBa-NL]

not sure if this would fix your issue.
but seems at least you should change the used name of the cookie from 
JSESSIONID to BIGipServervipf5-PRD

As currently your overwriting the session cookie the application needs.

Jorge Severino schreef op 6-3-2014 22:14:

Hi Guys:

I replaced one F5 BigIP for haproxy and everything works fine.

The only thing that not works for me is a weblogic cluster, if it 
works in F5.


I can help put the equivalent parameters?

I test with indirect cookie, prefix, direct, mode http, etc.

Send Off the curl query outputs to real server, virtual Ip by F5 and 
haproxy.



Direct Query a real server Weblogic:

[jseverino@jseverino Desktop]$ curl -v 
http://wls2:7101/BrowserWeb/servlet/BrowserServlet

* About to connect() to wls2 port 7101 (#0)
*   Trying x.x.x.x... connected
* Connected to wls2 (x.x.x.x) port 7101 (#0)
> GET /BrowserWeb/servlet/BrowserServlet HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 
NSS/3.14.0.0  zlib/1.2.3 libidn/1.18 libssh2/1.4.2

> Host: wls2:7101
> Accept: */*
>
< HTTP/1.1 200 OK
< Cache-Control: no-cache,no-store
< Date: Thu, 06 Mar 2014 21:07:53 GMT
< Pragma: no-cache
< Content-Length: 6825
< Content-Type: text/html;charset=utf-8
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
< X-FRAME-OPTIONS: SAMEORIGIN
< Set-Cookie: 
JSESSIONID=hGg6TYjJR82C0lhDFy5w9Qc50qlfDR74SKlHM3LykJj2XhtkMyHH!2087482587!1453328874; 
path=/; HttpOnly

< X-Powered-By: Servlet/3.0 JSP/2.2


Query a F5 VS:

[jseverino@jseverino Desktop]$ curl -v 
http://vipf5/BrowserWeb/servlet/BrowserServlet

* About to connect() to x.x.x.x port 80 (#0)
*   Trying x.x.x.x... connected
* Connected to x.x.x.x (x.x.x.x) port 80 (#0)
> GET /BrowserWeb/servlet/BrowserServlet HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 
NSS/3.14.0.0  zlib/1.2.3 libidn/1.18 libssh2/1.4.2

> Host: x.x.x.x
> Accept: */*
>
< HTTP/1.1 200 OK
< Cache-Control: no-cache,no-store
< Date: Thu, 06 Mar 2014 21:09:59 GMT
< Pragma: no-cache
< Content-Length: 6825
< Content-Type: text/html;charset=utf-8
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
< X-FRAME-OPTIONS: SAMEORIGIN
< Set-Cookie: 
JSESSIONID=nzLTTYkHGQ4h2gnB2hJc3tH3rqngyyhmtp2Jn4bS1LLrZ1SJMjb2!2087482587!1453328874; 
path=/; HttpOnly

< X-Powered-By: Servlet/3.0 JSP/2.2
< Set-Cookie: BIGipServervipf5-PRD=1992818698.48411.; path=/
< Vary: Accept-Encoding


And Query through Haproxy:

[jseverino@jseverino Desktop]$ curl -v 
http://haproxy/BrowserWeb/servlet/BrowserServlet

* About to connect() to haproxy port 80 (#0)
*   Trying x.x.x.x... connected
* Connected to haproxy (x.x.x.x) port 80 (#0)
> GET /BrowserWeb/servlet/BrowserServlet HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 
NSS/3.14.0.0  zlib/1.2.3 libidn/1.18 libssh2/1.4.2

> Host: haproxy
> Accept: */*
>
< HTTP/1.1 200 OK
< Cache-Control: no-cache,no-store
< Date: Thu, 06 Mar 2014 21:12:28 GMT
< Pragma: no-cache
< Content-Length: 6825
< Content-Type: text/html;charset=utf-8
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
< X-FRAME-OPTIONS: SAMEORIGIN
< X-Powered-By: Servlet/3.0 JSP/2.2
< Set-Cookie: JSESSIONID=server01; path=/


Thaks for helpme.

Atte.
Jorge Severino

Re: Equivalent Weblogic and BigIP

[Jorge Severino]

I will tomorrow test with users.

I replicated answer with.

Indirect preserve httponly
El 06/03/2014 18:34, "PiBa-NL"  escribió:

>  not sure if this would fix your issue.
> but seems at least you should change the used name of the cookie from
> JSESSIONID to BIGipServervipf5-PRD
> As currently your overwriting the session cookie the application needs.
>
> Jorge Severino schreef op 6-3-2014 22:14:
>
> Hi Guys:
>
>  I replaced one F5 BigIP for haproxy and everything works fine.
>
>  The only thing that not works for me is a weblogic cluster, if it works
> in F5.
>
>  I can help put the equivalent parameters?
>
>  I test with indirect cookie, prefix, direct, mode http, etc.
>
>  Send Off the curl query outputs to real server, virtual Ip by F5 and
> haproxy.
>
>
>  Direct Query a real server Weblogic:
>
>  [jseverino@jseverino Desktop]$ curl -v
> http://wls2:7101/BrowserWeb/servlet/BrowserServlet
> * About to connect() to wls2 port 7101 (#0)
> *   Trying x.x.x.x... connected
> * Connected to wls2 (x.x.x.x) port 7101 (#0)
> > GET /BrowserWeb/servlet/BrowserServlet HTTP/1.1
> > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/
> 3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> > Host: wls2:7101
> > Accept: */*
> >
> < HTTP/1.1 200 OK
> < Cache-Control: no-cache,no-store
> < Date: Thu, 06 Mar 2014 21:07:53 GMT
> < Pragma: no-cache
> < Content-Length: 6825
> < Content-Type: text/html;charset=utf-8
> < Expires: Thu, 01 Jan 1970 00:00:00 GMT
> < X-FRAME-OPTIONS: SAMEORIGIN
> < Set-Cookie:
> JSESSIONID=hGg6TYjJR82C0lhDFy5w9Qc50qlfDR74SKlHM3LykJj2XhtkMyHH!2087482587!1453328874;
> path=/; HttpOnly
> < X-Powered-By: Servlet/3.0 JSP/2.2
>
>
>  Query a F5 VS:
>
>  [jseverino@jseverino Desktop]$ curl -v
> http://vipf5/BrowserWeb/servlet/BrowserServlet
> * About to connect() to x.x.x.x port 80 (#0)
> *   Trying x.x.x.x... connected
> * Connected to x.x.x.x (x.x.x.x) port 80 (#0)
> > GET /BrowserWeb/servlet/BrowserServlet HTTP/1.1
>  > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/
> 3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> > Host: x.x.x.x
> > Accept: */*
> >
> < HTTP/1.1 200 OK
> < Cache-Control: no-cache,no-store
> < Date: Thu, 06 Mar 2014 21:09:59 GMT
> < Pragma: no-cache
> < Content-Length: 6825
> < Content-Type: text/html;charset=utf-8
> < Expires: Thu, 01 Jan 1970 00:00:00 GMT
> < X-FRAME-OPTIONS: SAMEORIGIN
> < Set-Cookie:
> JSESSIONID=nzLTTYkHGQ4h2gnB2hJc3tH3rqngyyhmtp2Jn4bS1LLrZ1SJMjb2!2087482587!1453328874;
> path=/; HttpOnly
> < X-Powered-By: Servlet/3.0 JSP/2.2
> < Set-Cookie: BIGipServervipf5-PRD=1992818698.48411.; path=/
> < Vary: Accept-Encoding
>
>
>  And Query through Haproxy:
>
>  [jseverino@jseverino Desktop]$ curl -v
> http://haproxy/BrowserWeb/servlet/BrowserServlet
> * About to connect() to haproxy port 80 (#0)
> *   Trying x.x.x.x... connected
> * Connected to haproxy (x.x.x.x) port 80 (#0)
> > GET /BrowserWeb/servlet/BrowserServlet HTTP/1.1
> > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/
> 3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> > Host: haproxy
> > Accept: */*
> >
> < HTTP/1.1 200 OK
> < Cache-Control: no-cache,no-store
> < Date: Thu, 06 Mar 2014 21:12:28 GMT
> < Pragma: no-cache
> < Content-Length: 6825
> < Content-Type: text/html;charset=utf-8
> < Expires: Thu, 01 Jan 1970 00:00:00 GMT
> < X-FRAME-OPTIONS: SAMEORIGIN
> < X-Powered-By: Servlet/3.0 JSP/2.2
> < Set-Cookie: JSESSIONID=server01; path=/
>
>
>  Thaks for helpme.
>
>  Atte.
> Jorge Severino
>
>
>

Fwd: rewrite URI help

[Steve Phillips]

Haven't gotten a response on this...trying again...any help appreciated.


Trying to reverse proxy all requests to

/slideshare

to

www.slideshare.net/api/2/get_slideshow

my front-end config:

 acl url_slideshare   path_dir   slideshare
 use_backend slideshare if url_slideshare

and back-end:

backend slideshare
  option http-server-close
  option httpclose
  reqrep ^([^\ ]*)\ /slideshare(.*)  \1\ /api/2/get_slideshow\2
  server slideshare www.slideshare.net:443 ssl verify none

requests to /slideshow however, are not being rewritten:

173.11.67.214:60821 [04/Mar/2014:19:49:03.257] main slideshare/slideshare
6142/0/289/121/6552 404 9299 - -  0/0/0/0/0 0/0 {} "GET
/slideshare?slideshow_url=http%3A%2F%2Fwww.slideshare.net%2FAaronKlein1%2Foptimizing-aws-economics&detailed=1&api_key=msCpLON8&hash=a7fe5fd52cc86e4a4a3d1022cb7c63476b79e044&ts=1393980574
HTTP/1.1"

Is my regex incorrect?  Am I missing something else?

Thanks.

Steve

Re: rewrite URI help

[Bryan Talbot]

On Thu, Mar 6, 2014 at 1:42 PM, Steve Phillips  wrote:

> Haven't gotten a response on this...trying again...any help appreciated.
>
>
> Trying to reverse proxy all requests to
>
> /slideshare
>


slideSHARE



>
> to
>
> www.slideshare.net/api/2/get_slideshow
>
> my front-end config:
>
>  acl url_slideshare   path_dir   slideshare
>  use_backend slideshare if url_slideshare
>
> and back-end:
>
> backend slideshare
>   option http-server-close
>   option httpclose
>   reqrep ^([^\ ]*)\ /slideshare(.*)  \1\ /api/2/get_slideshow\2
>   server slideshare www.slideshare.net:443 ssl verify none
>
> requests to /slideshow however, are not being rewritten:
>


slideSHOW

What do you want to rewrite, SHARE or SHOW?

-Bryan

Re: rewrite URI help

[Baptiste]

Hi Steve,

run HAProxy in debug mode and compare the request received and the
request sent and you'll know if you have an error.

At first sight, your rule looks correct.

Baptiste


On Thu, Mar 6, 2014 at 10:42 PM, Steve Phillips  wrote:
> Haven't gotten a response on this...trying again...any help appreciated.
>
>
> Trying to reverse proxy all requests to
>
> /slideshare
>
> to
>
> www.slideshare.net/api/2/get_slideshow
>
> my front-end config:
>
>  acl url_slideshare   path_dir   slideshare
>  use_backend slideshare if url_slideshare
>
> and back-end:
>
> backend slideshare
>   option http-server-close
>   option httpclose
>   reqrep ^([^\ ]*)\ /slideshare(.*)  \1\ /api/2/get_slideshow\2
>   server slideshare www.slideshare.net:443 ssl verify none
>
> requests to /slideshow however, are not being rewritten:
>
> 173.11.67.214:60821 [04/Mar/2014:19:49:03.257] main slideshare/slideshare
> 6142/0/289/121/6552 404 9299 - -  0/0/0/0/0 0/0 {} "GET
> /slideshare?slideshow_url=http%3A%2F%2Fwww.slideshare.net%2FAaronKlein1%2Foptimizing-aws-economics&detailed=1&api_key=msCpLON8&hash=a7fe5fd52cc86e4a4a3d1022cb7c63476b79e044&ts=1393980574
> HTTP/1.1"
>
> Is my regex incorrect?  Am I missing something else?
>
> Thanks.
>
> Steve
>

Re: Fwd: rewrite URI help

[Cyril Bonté]

Hi,

Le 06/03/2014 22:42, Steve Phillips a écrit :

Haven't gotten a response on this...trying again...any help appreciated.


Patrick Hemmer already replied to the list yesterday (but without 
including your email address).




Trying to reverse proxy all requests to

/slideshare

to

www.slideshare.net/api/2/get_slideshow


my front-end config:

  acl url_slideshare   path_dir   slideshare
  use_backend slideshare if url_slideshare

and back-end:

backend slideshare
   option http-server-close
   option httpclose
   reqrep ^([^\ ]*)\ /slideshare(.*)  \1\ /api/2/get_slideshow\2
   server slideshare www.slideshare.net:443
 ssl verify none

requests to /slideshow however, are not being rewritten:


uh ? Why would you want /slideshow to be rewritten if your regex is 
designed for /slideshare ?




173.11.67.214:60821 
[04/Mar/2014:19:49:03.257] main slideshare/slideshare
6142/0/289/121/6552 404 9299 - -  0/0/0/0/0 0/0 {} "GET
/slideshare?slideshow_url=http%3A%2F%2Fwww.slideshare.net
%2FAaronKlein1%2Foptimizing-aws-economics&detailed=1&api_key=msCpLON8&hash=a7fe5fd52cc86e4a4a3d1022cb7c63476b79e044&ts=1393980574
HTTP/1.1"

Is my regex incorrect?  Am I missing something else?


The regex is valid, it depends on what you *really* want to do.
As Patrick said :
"
The haproxy log contains the original request, not the rewritten one. If 
you want to see the rewritten URL you need to look at the backend server 
which is receiving the request.


-Patrick
"

--
Cyril Bonté

Re: Fwd: rewrite URI help

[Steve Phillips]

I can't look at the backend server as it is slideshare.net.

To answer your question Cyril...requests coming to the front-end will be of
the form:

/slideshare...

I want to rewrite all instances of /slideshare -> /api/2/get_slideshow

and then reverse proxy the result to www.slideshare.net:

/slideshare -> www.slideshare.net/api/2/get_slideshow





On Thu, Mar 6, 2014 at 2:04 PM, Cyril Bonté  wrote:

> Hi,
>
> Le 06/03/2014 22:42, Steve Phillips a écrit :
>
>  Haven't gotten a response on this...trying again...any help appreciated.
>>
>
> Patrick Hemmer already replied to the list yesterday (but without
> including your email address).
>
>
>> Trying to reverse proxy all requests to
>>
>> /slideshare
>>
>> to
>>
>> www.slideshare.net/api/2/get_slideshow
>> 
>>
>>
>> my front-end config:
>>
>>   acl url_slideshare   path_dir   slideshare
>>   use_backend slideshare if url_slideshare
>>
>> and back-end:
>>
>> backend slideshare
>>option http-server-close
>>option httpclose
>>reqrep ^([^\ ]*)\ /slideshare(.*)  \1\ /api/2/get_slideshow\2
>>server slideshare www.slideshare.net:443
>>  ssl verify none
>>
>>
>> requests to /slideshow however, are not being rewritten:
>>
>
> uh ? Why would you want /slideshow to be rewritten if your regex is
> designed for /slideshare ?
>
>
>> 173.11.67.214:60821 
>>
>> [04/Mar/2014:19:49:03.257] main slideshare/slideshare
>> 6142/0/289/121/6552 404 9299 - -  0/0/0/0/0 0/0 {} "GET
>> /slideshare?slideshow_url=http%3A%2F%2Fwww.slideshare.net
>> %2FAaronKlein1%2Foptimizing-
>> aws-economics&detailed=1&api_key=msCpLON8&hash=
>> a7fe5fd52cc86e4a4a3d1022cb7c63476b79e044&ts=1393980574
>>
>> HTTP/1.1"
>>
>> Is my regex incorrect?  Am I missing something else?
>>
>
> The regex is valid, it depends on what you *really* want to do.
> As Patrick said :
> "
> The haproxy log contains the original request, not the rewritten one. If
> you want to see the rewritten URL you need to look at the backend server
> which is receiving the request.
>
> -Patrick
> "
>
> --
> Cyril Bonté
>

Aumente el rendimiento de su equipo

[Katherine gonzalez]

Cómo Supervisar al Personal… Técnicas efectivas de liderazgo para 
maximizar el rendimiento de su equipoCali 19 
- Bogotá 20 
- Medellín 
20 de Marzo de 
2014


  
  

  Usted 
  se llevará valiosa información que le servirá de referencia una y otra 
  vez, obtendrá mayor confianza en sí mismo y en su rol de Supervisor y a la 
  vez reducirá el estrés y los aspectos negativos de su trabajo. 
  Para ampliar 
  la información y obtener los beneficios de inscripción temprana diligencie 
  sin compromiso los siguientes datos: 
  -Nombre:-Empresa:-Ciudad:-Teléfono:-E-mail: 
  haproxy@formilux.org"Su información jamás será compartida ni comercializada. 
  Garantizamos total confidencialidad y privacidad de sus 
  datos"
  

  Centro de 
  atención telefónica: 01 8000 51 30 51, PBX (4) 444 09 18 
  |
  

  

  Importante: En 
  cumplimiento con la ley 1581 de 2012, queremos comunicarle que si usted no 
  desea recibir la información actualizada con los temas más innovadores de 
  nuestra agenda de eventos de capacitación, puede des-suscribirse de estas 
  invitaciones respondiendo este correo con el asunto BAJA.Este 
  correo no puede ser considerado intrusivo ya que cumple con las políticas 
  antispa m internacionales y locales.Este correo ha sido enviado 
  enviado a: haproxy@formilux.org

Re: Fwd: rewrite URI help

[Cyril Bonté]

Le 06/03/2014 23:47, Steve Phillips a écrit :

I can't look at the backend server as it is slideshare.net
.


You can introduce a "debug" proxy in your configuration to log what 
happens after the reqrep.



To answer your question Cyril...requests coming to the front-end will be
of the form:

/slideshare...

I want to rewrite all instances of /slideshare -> /api/2/get_slideshow

and then reverse proxy the result to www.slideshare.net
:

/slideshare -> www.slideshare.net/api/2/get_slideshow



Then it is already working.
Maybe you need to rewrite the Host header, also. Add this line in your 
backend (I assume you're using a recent 1.5-dev version, otherwise I let 
you guess the other way to do it with older keywords) :

  http-request set-header Host www.slideshare.net

One mystery remains because you didn't provide a complete configuration.
Does your "main" frontend use multiple backends depending on acls ? 
Maybe there's a configuration issue in one of the backends or directly 
in the frontend (for example because the tunnel mode is used for other 
accesses than /slideshare).


Btw, don't mix "option http-server-close" and "option httpclose".

--
Cyril Bonté

Re: Fwd: rewrite URI help

[Steve Phillips]

Yes, using the latest 1.5-dev version.  Was going to paste my config but no
need.  The set-header  option solved the problem.  Thank you again Cyril.
 I guess I owe you several beers when you visit San Francisco:)

Steve


On Thu, Mar 6, 2014 at 3:10 PM, Cyril Bonté  wrote:

> Le 06/03/2014 23:47, Steve Phillips a écrit :
>
>> I can't look at the backend server as it is slideshare.net
>> .
>>
>
> You can introduce a "debug" proxy in your configuration to log what
> happens after the reqrep.
>
>  To answer your question Cyril...requests coming to the front-end will be
>> of the form:
>>
>> /slideshare...
>>
>> I want to rewrite all instances of /slideshare -> /api/2/get_slideshow
>>
>> and then reverse proxy the result to www.slideshare.net
>> :
>>
>>
>> /slideshare -> www.slideshare.net/api/2/get_slideshow
>> 
>>
>
> Then it is already working.
> Maybe you need to rewrite the Host header, also. Add this line in your
> backend (I assume you're using a recent 1.5-dev version, otherwise I let
> you guess the other way to do it with older keywords) :
>   http-request set-header Host www.slideshare.net
>
> One mystery remains because you didn't provide a complete configuration.
> Does your "main" frontend use multiple backends depending on acls ? Maybe
> there's a configuration issue in one of the backends or directly in the
> frontend (for example because the tunnel mode is used for other accesses
> than /slideshare).
>
> Btw, don't mix "option http-server-close" and "option httpclose".
>
> --
> Cyril Bonté
>

Possibly redundant code

[Sasha Pachev]

I am looking at this code in src/proto_http.c in the development tree:

case HTTP_REQ_ACT_ADD_HDR:
chunk_printf(&trash, "%s: ", rule->arg.hdr_add.name);
memcpy(trash.str, rule->arg.hdr_add.name, rule->arg.hdr_add.name_len);
trash.len = rule->arg.hdr_add.name_len;
trash.str[trash.len++] = ':';
trash.str[trash.len++] = ' ';
trash.len += build_logline(s, trash.str + trash.len, trash.size -
trash.len, &rule->arg.hdr_add.fmt);
http_header_add_tail2(&txn->req, &txn->hdr_idx, trash.str, trash.len);

Looks like chunk_printf() call is redundant as it does the same thing
as memcpy() + manual trash.str fixing later on.


-- 
Sasha Pachev

Fast Running Blog.
http://fastrunningblog.com
Run. Blog. Improve. Repeat.

Proxying https to https

[Mike Goff]

Hi,

I’m trying to figure out how to do the equivalent of this working nginx sample 
configuration using HAProxy 1.5-dev22:

server {
  listen 443 ssl default;
  ssl_certificate /etc/nginx/certs/cloud.appcelerator.com.combined.crt;
  ssl_certificate_key /etc/nginx/certs/cloud.appcelerator.com.key;

  location / {
proxy_pass https://www.google.com:443;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_next_upstream error;
proxy_redirect off;
  }
}

HAProxy documentation such as 
http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#4.2-stick%20store-response
 indicates setting the backend mode to tcp:

global
maxconn 4000
daemon

defaults
timeout server 5s
timeout connect 5s
timeout client 5s

frontend proxy-secure
bind :443 ssl crt /etc/haproxy/cloud.appcelerator.com.pem
reqadd X-Forwarded-Proto:\ https
default_backend https

backend https
mode tcp
server google-secure www.google.com:443

However, this results in an empty response:

$ curl -v https://haproxy-ssl-test.cloud.appcelerator.com
* Adding handle: conn: 0x7f7fea003a00
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7f7fea003a00) send_pipe: 1, recv_pipe: 0
* About to connect() to 
haproxy-ssl-test.cloud.appcelerator.com
 port 443 (#0)
*   Trying 54.203.245.228...
* Connected to 
haproxy-ssl-test.cloud.appcelerator.com
 (54.203.245.228) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
* Server certificate: *.cloud.appcelerator.com
* Server certificate: Go Daddy Secure Certificate Authority - G2
* Server certificate: Go Daddy Root Certificate Authority - G2
* Server certificate: Go Daddy Class 2 Certification Authority
> GET / HTTP/1.1
> User-Agent: curl/7.30.0
> Host: 
> haproxy-ssl-test.cloud.appcelerator.com
> Accept: */*
>
* Empty reply from server
* Connection #0 to host 
haproxy-ssl-test.cloud.appcelerator.com
 left intact
curl: (52) Empty reply from server

So I think that I am configuring the backend incorrectly. I can get https -> 
http proxying working just fine, but I need to be able to proxy (not redirect) 
to an https endpoint. Any suggestions are greatly appreciated.

thanks!
Mike

--
Michael Goff
Director of Engineering, DevOps

Appcelerator, Inc.
The Next Generation Mobile Platform

Re: Proxying https to https

[Nenad Merdanovic]

Hello Mike,

On 03/07/2014 01:55 AM, Mike Goff wrote:
> Hi,
> 
> I’m trying to figure out how to do the equivalent of this working nginx
> sample configuration using HAProxy 1.5-dev22:
> [...] 
> frontend proxy-secure
> bind :443 ssl crt /etc/haproxy/cloud.appcelerator.com.pem
> reqadd X-Forwarded-Proto:\ https
> default_backend https
> 
> backend https
> mode tcp
> server google-secure www.google.com:443
> 
> However, this results in an empty response:
> 
> $ curl -v https://haproxy-ssl-test.cloud.appcelerator.com

You need to add the 'ssl' keyword to the server line in the backend.
Then you need to either load the CA certificates used for verification
or add 'verify none'. Something like this should do:

backend https
mode http
server google-secure www.google.com:443 ssl verify none

Regards,
-- 
Nenad Merdanovic | PGP: 0x423edcb2 | Web: http://nimzo.info
Linkedin: http://www.linkedin.com/in/nenadmerdanovic

Re: Proxying https to https

[Mike Goff]

Hi Nenad,

That works perfectly. Thanks so much for your help!

Mike

--
Michael Goff
Director of Engineering, DevOps



Appcelerator, Inc.
The Next Generation Mobile Platform



On Mar 6, 2014, at 5:36 PM, Nenad Merdanovic 
mailto:ni...@nimzo.info>> wrote:

Hello Mike,

On 03/07/2014 01:55 AM, Mike Goff wrote:
Hi,

I’m trying to figure out how to do the equivalent of this working nginx
sample configuration using HAProxy 1.5-dev22:
[...]
frontend proxy-secure
   bind :443 ssl crt /etc/haproxy/cloud.appcelerator.com.pem
   reqadd X-Forwarded-Proto:\ https
   default_backend https

backend https
   mode tcp
   server google-secure www.google.com:443

However, this results in an empty response:

$ curl -v https://haproxy-ssl-test.cloud.appcelerator.com

You need to add the 'ssl' keyword to the server line in the backend.
Then you need to either load the CA certificates used for verification
or add 'verify none'. Something like this should do:

backend https
   mode http
   server google-secure www.google.com:443 ssl verify 
none

Regards,
--
Nenad Merdanovic | PGP: 0x423edcb2 | Web: http://nimzo.info
Linkedin: http://www.linkedin.com/in/nenadmerdanovic

Re: Possibly redundant code

[Willy Tarreau]

Hi Sasha,

On Thu, Mar 06, 2014 at 05:53:30PM -0700, Sasha Pachev wrote:
> I am looking at this code in src/proto_http.c in the development tree:
> 
> case HTTP_REQ_ACT_ADD_HDR:
> chunk_printf(&trash, "%s: ", rule->arg.hdr_add.name);
> memcpy(trash.str, rule->arg.hdr_add.name, rule->arg.hdr_add.name_len);
> trash.len = rule->arg.hdr_add.name_len;
> trash.str[trash.len++] = ':';
> trash.str[trash.len++] = ' ';
> trash.len += build_logline(s, trash.str + trash.len, trash.size -
> trash.len, &rule->arg.hdr_add.fmt);
> http_header_add_tail2(&txn->req, &txn->hdr_idx, trash.str, trash.len);
> 
> Looks like chunk_printf() call is redundant as it does the same thing
> as memcpy() + manual trash.str fixing later on.

Ah yes you're right. I just checked the history to see if one was added
before the other one or whatever, and no, both were merged with the
feature at the same time in commit 20b0de56.

I suspect that I first used chunk_printf() and realized that it would
be faster done by hand since we had everything for this, and that I
forgot to remove the chunk_printf(). Feel free to send a patch to
remove it.

Thanks,
Willy

Re: Environment variables in port part of bind?

[Willy Tarreau]

On Thu, Mar 06, 2014 at 02:08:07PM -0600, a...@fsf.net wrote:
>From the docs, it appears that HAProxy does not support environment
>variables in the port portion of the bind parameter.  Is this correct?  If
>it is correct, is there a reason for this choice, or is it just that no one
>has asked for the feature before?

I think this comment is wrong now. I've just reread the code. Addresses
are resolved by str2sa_range(). It supports addresses, port ranges and
even family prefixes (eg: "ipv6@"). And this function first calls
env_expand() to expand any "${xxx}" present on the input string. So any
address can have any part being retrieved from environment variables.

If you found some misleading parts in the doc about this, feel free to
amend them and propose a patch for inclusion.

Thanks!
Willy

Re: Current solutions to the soft-restart-healthcheck-spread problem?

[Willy Tarreau]

Hi Jonathan,

On Thu, Mar 06, 2014 at 03:15:25PM +, Jonathan Matthews wrote:
> Hi all -
> 
> [ tl;dr How do you stop haproxy using failed backend servers immediately
> after reload?
> Haproxy devs, please consider implementing a
> consider-servers-initially-DOWN option! ]
> 
> I wonder if people could outline how they're dealing with the combination
> of these two haproxy behaviours:
> 
> 1) On restart/reload/disabled-server-now-enabled-via-admin-interface,
> haproxy considers a server to be 1 health check away from going down, but
> considers it *initially* up.
> 
> 2) On restart/reload, haproxy spreads out each backend's(?) initial server
> health checks over the entire health check interval.
> 
> (If I'm slightly off with either of those statements, please forgive the
> inaccuracy and let it slide for the purposes of this discussion; do let me
> know if I'm /meaningfully/ wrong of course!)
> 
> The combination of these facts in a high traffic environment seems to imply
> that an unhealthy-but-just-enabled server which is listed last in an
> haproxy backend may receive requests for a longer-than-expected period of
> time, resulting in a non-trivial number of requests failing.
> 
> In such an environment, where multiple load balancers are involved and can
> be reloaded sequentially (such as mine!), it would be preferable to take a
> pessimistic approach and /not/ expose servers to traffic until you're
> positive that the backend is healthy, rather than haproxy's current
> default-optimism approach.
> 
> I've been considering some methods to deal with this, but haven't got a
> working config yet. It's getting somewhat convoluted and stick-table heavy,
> so I thought I'd ask everyone:
> 
> Where you have decided that this is something you actually need to deal
> with, *how* are you doing that? (I totally recognise that the combination
> of a frequent health check interval and non-insane traffic volumes may mask
> this issue, leading many -- myself included in previous jobs! -- not to
> consider it a problem in the first place)
> 
> It's worth pointing out that I /believe/ this situation could be easily
> solved (operationally) by a global, per-backend or per-server option which
> switches on the pessimistic behaviour mentioned above. I recognise that
> this may not be easy from an /implementation/ perspective, of course.
> [Willy: any chance of an option to start each server as if it were down,
> but being 1 check away from going up, rather than the opposite? :-)]

I'm adding this to the todo list. In fact, this mode was chosen more than
10 years ago after having been forced to live with equipments doing the
exact opposite (what you're asking for) : you start the equipment, it
receives traffic and drops everything because no server is up yet. With
the current behaviour, even if you have one dead server in the farm, the
server is properly distributed to valid servers and the dead server
causes a redispatch after a few retries.

But I agree, we need to have options to start up by default or down by
default.

Concerning the start of health checks, I'm now thinking that we could
have a global parameter indicating the maximum distance between the
first and the last health check. It would probably satisfy all users.

> It's also worth pointing out that, whilst the "persist haproxy state over
> soft restarts" concept that's been mentioned previously on list would solve
> this for orderly restarts, it wouldn't solve it for crashes, reboots or
> otherwise. I think the option I mentioned above would be one way to solve
> it nicely, for multiple use cases.

Yes but nobody had the time to work on it yet. I just wanted to have the
ability to send a state dump to a file (eg: "show servers" on the CLI)
and feed this format on the input of the new process. It would be very
simple to do and very efficient.

> [ For a *not* nice solution, I'll post a follow up when I get my
> stick-table concept going. It's /nasty/. IMHO. Don't make me put it into
> production! ;-) ]

OK

cheers,
Willy

Stats for total queued backend requests

[Laurent Dormoy]

Hi,

I have a hard time collecting statistics for queued backend requests. It 
usually happens during a couple of seconds and cannot be efficiently tracked by 
a monitoring system collecting data at regular intervals.

Is it possible to add to the stats page/csv a counter for total queued requests 
(like the total sessions counter) ?


Cheers,


Laurent