Re: help for setting up haproxy using Linux system proxy

[Vivek Malik]

I had faced the same issue and have documented my solution at
https://www.mail-archive.com/haproxy@formilux.org/msg26387.html

Regards,
Vivek

On Sat, Jul 8, 2017 at 5:44 AM, Aaron West  wrote:
> On 8 July 2017 at 11:25, Aaron West  wrote:
>> for me at least, it would
>> boggle the mind that you have a reverse proxy using a forward proxy.
>
> I think I I should clarify the above... I don't mean using forward
> proxies as a real server(Think webfilters) because that makes sense I
> just mean routing to client or server via a forward proxy seems
> confusing to me...
>
> Aaron West
>
> Loadbalancer.org
>
> www.loadbalancer.org
> +1 888 867 9504 / +44 (0)330 380 1064
> aa...@loadbalancer.org
>
> LEAVE A REVIEW | DEPLOYMENT GUIDES | BLOG
>

Re: squid between haproxy and backend server

[Vivek Malik]

Update:

I was able to solve for this by using Apache ProxyRemote Option. Here
is the working test configuration of Apache.


SSLProxyEngine On
SSLProxyCheckPeerCN on
SSLProxyCheckPeerExpire on
RewriteRule / https://remote.domain [P]
ProxyRemote "*" https://proxyIp:proxyPort
ProxyPass / https://remote.domain


The POC configuration which worked for me is

Client => Haproxy => Apache => Squid => ELB

Unlike what many blogs listed, ProxyRemote worked without ProxyRequests on.

I tried researching lua with haproxy, but I couldn't figure out if I
can force a CONNECT via lua to the backend before haproxy starts
forwarding the request to backend.

Regards,
Vivek

On Sun, Jun 11, 2017 at 11:22 PM, Vivek Malik <vivek.ma...@gmail.com> wrote:
> Hi,
>
> I am using haproxy for path based routing and am trying to forward
> traffic to another datacenter under certain conditions. However, the
> traffic must go via explicit proxy server (squid). The traffic is
> getting forwarded to an https endpoint in AWS fronted by ELB.
>
> Is there a way to define "via proxy" in backend configuration? I did
> configuration search, mailing list archive search and google search
> but couldn't come up with anything.
>
> Or is there any software which can forward traffic via squid to
> another endpoint. (I checked stunnel but didn't find such feature)
>
> Thanks for guiding me with ideas or pointers.
>
> Regards,
> Vivek

squid between haproxy and backend server

[Vivek Malik]

Hi,

I am using haproxy for path based routing and am trying to forward
traffic to another datacenter under certain conditions. However, the
traffic must go via explicit proxy server (squid). The traffic is
getting forwarded to an https endpoint in AWS fronted by ELB.

Is there a way to define "via proxy" in backend configuration? I did
configuration search, mailing list archive search and google search
but couldn't come up with anything.

Or is there any software which can forward traffic via squid to
another endpoint. (I checked stunnel but didn't find such feature)

Thanks for guiding me with ideas or pointers.

Regards,
Vivek

Is www.haproxy.org down?

[Vivek Malik]

Hi,

I am getting 500 Internal Server Error on the haproxy.org website.
Various websites like

http://downforeveryoneorjustme.com/www.haproxy.org
http://www.isitdownrightnow.com/haproxy.org.html
http://www.websitenotworking.com/haproxy.org

confirm that the website is down for them too.

Regards,
Vivek

bug? rand based acl keep re-evaluating

[Vivek Malik]

Hi,

I spent last few hours scratching my head and wondering where my
configuration was wrong and why was it acting weird. Sending this to
ML to see if this is a bug or expected behavior. I expected ACL to
evaluate its condition only once and store the true/false outcome,
however with rand(), I believe that ACL keeps re-evaluating on every
use. Example,

acl random rand(50) lt 25
http-response set-header H1 V1 if random
http-response set-header H2 V2 if random
http-response set-header H3 V3 if random

I would except either all 3 headers H1, H2, H3 to be set or none to be
set. However, in my test config I observed that H1, H2, H3 are set
independent of each other essentially meaning that ACL is getting
re-evaluated every time it is getting used.

Is this the expected behavior of ACLs? If yes, how would using a named
ACL be different than using an anonymous ACL?

Regards,
Vivek

Re: http-request set-map key as fixed string

[Vivek Malik]

(Sorry for pressing send too early)

Hi,

I am trying to use http-request set-map with fixed strings which are
defined in config file. Example lines

http-request set-map(motion.map) monday %[date()] if acl_1
http-request set-map(motion.map) tuesday %[date()] if acl_2

Note that monday and tuesday are literal string and not log-format
variables. However, when I look into the map using

echo show map motion.map | socat stdio /var/run/socket-haproxy
0x13c1b90  1425089705

As you can see, the key is missing from the map. monday is getting
replaced by space  

However, if I switch the format to
http-request set-map(motion.map) %[date()] monday if acl_1

such that the literal string monday is a value instead of key, the
set-map works fine.

echo show map motion.map | socat stdio /var/run/socket-haproxy
0x13c1b90 1425089710 monday


Please suggest if I have stumbled across a bug or I am missing
something in my configuration.

Regards,
Vivek

On Fri, Feb 27, 2015 at 8:20 PM, Vivek Malik vivek.ma...@gmail.com wrote:
 Hi,

 I am trying to use http-request set-map with fixed strings which are
 defined in config file. Example lines

 http-request set-map(motion.map) monday %[date()] if acl_1
 http-request set-map(motion.map) tuesday %[date()] if acl_2

 Note that monday and tuesday are literal string and not log-format
 variables. However, when I look into the map using

 echo show map motion.map | socat stdio /var/run/socket-haproxy
 0x13c1b90  1425089705

 As you can see, the key is missing from the map. monday is getting
 replaced by space  

 However, if I switch the format to

Re: http-request set-map key as fixed string

[Vivek Malik]

Hi Baptise,

Using set-map on the stats socket gives the expected result (except
that I can't use functions there). set map motion.map monday 12345 did
set the map with key monday and value as 12345.

I found that http-request set-map(motion.map) monday %[date] doesn't
work, but http-request set-map(motion.map) %t does work. Using %T,
%TL, %t works as expected but %[date] doesn't.

Moreover, I am getting crazy results when I try to use http-request
set-map(motion.map) monday %[src]. Using this makes both key and
value as src.

BTW, I am using

HA-Proxy version 1.5.11 2015/01/31
Copyright 2000-2015 Willy Tarreau w...@1wt.eu

Build options :
  TARGET  = linux2628
  CPU = generic
  CC  = gcc
  CFLAGS  = -O2 -g -fno-strict-aliasing
  OPTIONS = USE_ZLIB=1 USE_OPENSSL=1 USE_STATIC_PCRE=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200

Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.8
Compression algorithms supported : identity, deflate, gzip
Built with OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014
Running on OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 8.31 2012-07-06
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with transparent proxy support using: IP_TRANSPARENT
IPV6_TRANSPARENT IP_FREEBIND

Available polling systems :
  epoll : pref=300,  test result OK
   poll : pref=200,  test result OK
 select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

On Sat, Feb 28, 2015 at 1:26 AM, Baptiste bed...@gmail.com wrote:
 On Sat, Feb 28, 2015 at 3:22 AM, Vivek Malik vivek.ma...@gmail.com wrote:
 (Sorry for pressing send too early)

 Hi,

 I am trying to use http-request set-map with fixed strings which are
 defined in config file. Example lines

 http-request set-map(motion.map) monday %[date()] if acl_1
 http-request set-map(motion.map) tuesday %[date()] if acl_2

 Note that monday and tuesday are literal string and not log-format
 variables. However, when I look into the map using

 echo show map motion.map | socat stdio /var/run/socket-haproxy
 0x13c1b90  1425089705

 As you can see, the key is missing from the map. monday is getting
 replaced by space  

 However, if I switch the format to
 http-request set-map(motion.map) %[date()] monday if acl_1

 such that the literal string monday is a value instead of key, the
 set-map works fine.

 echo show map motion.map | socat stdio /var/run/socket-haproxy
 0x13c1b90 1425089710 monday


 Please suggest if I have stumbled across a bug or I am missing
 something in my configuration.



 Hi Vivek,

 could you try the 'set-map' on the stats socket directly and report if
 you have the same result or not?

 Baptiste

http-request set-map key as fixed string

[Vivek Malik]

Hi,

I am trying to use http-request set-map with fixed strings which are
defined in config file. Example lines

http-request set-map(motion.map) monday %[date()] if acl_1
http-request set-map(motion.map) tuesday %[date()] if acl_2

Note that monday and tuesday are literal string and not log-format
variables. However, when I look into the map using

echo show map motion.map | socat stdio /var/run/socket-haproxy
0x13c1b90  1425089705

As you can see, the key is missing from the map. monday is getting
replaced by space  

However, if I switch the format to

Feature requests: set-cookie dynamic value / urlencode, urldecode

[Vivek Malik]

Hi,

I am trying to redirect a user to login page if not logged in and
redirect the user back to original page after login. The login page is
on a different domain than the primary domain where requests are
coming.

I have a few questions/feature requests related to http-request redirect

a) urlencode function - I want to capture the request uri in url
parameter while using http-request redirect. However, I can't seem to
find a way to encode the value

Something like,

http-request redirect code 302 location
http://sso.domain/login?referer=%[urlencode(capture.req.uri)]

I can use capture.req.uri only but that leads to an invalid url since
the value is not url encoded.

b) urldecode function - Post successful login, I would like to
redirect user back to original referer url

http-request redirect code 302 location %[urldecode(urlp(referer))]

c) set-cookie dynamic value - Based on my tests, set-cookie parameter
doesn't take dynamic values. I would like to set the value based on
request parameters

http-request redirect set-cookie sessionId=%[urlp(sessionId)] location

Please let me know if you see value in implementing these features in haproxy.

Regards,
Vivek

Re: Problems about Hot Configuration of Haproxy

[Vivek Malik]

Please see http://www.mail-archive.com/haproxy@formilux.org/msg06885.html

The summary being

iptables -I INPUT -p tcp --dport $PORT --syn -j DROP
sleep 1
service haproxy restart
iptables -D INPUT -p tcp --dport $PORT --syn -j DROP

Regards,
Vivek

On Tue, Jan 20, 2015 at 1:11 AM, hu.zhang hu.zh...@dev.bessystem.com wrote:
 Hi,Willy:

  I am a beginner of Haproxy. Recently I get a problem that hot
 configuration of haproxy still lead to failed request. I have read the
 answer from
 http://stackoverflow.com/questions/21595534/hot-reconfiguration-of-haproxy-still-lead-to-failed-request-any-suggestions.
 But the failed requests still exist when I use ApacheBench. Could you please
 give me some suggestion about the following two question?

 1.   Can I add or remove servers in backends without restarting a
 process? If I don’t start a haproxy process with –sf keyword, is there a way
 to add/remove the servers in real time?

 2.   If the answer of the the first question is no , Can you give me
 some tips about how to modify the source to achieve it?

 Thanks!

  With my best wishes,

  Hu.Zhang

Re: Round Robin not very random

[Vivek Malik]

I see roubdrobin working perfectly over here. Look at sessions total and
see how they are same for every server.

It seems that all your requests are not the same workload. Some servers or
some requests are taking longer to fulfill and increasing load on servers.
Have you tried using leastconn instead of round robin?

That might give more fair distribution of load in this situation.

Regards,
Vivek
On Jan 14, 2015 11:45 PM, Alexey Zilber alexeyzil...@gmail.com wrote:

 Hi All,

   We got hit with a bit of traffic and we saw haproxy dump most of the
 traffic to 3-4 app servers, sometimes even just one and driving load on
 there to 90.  We were running 1.5.9, I upgraded to 1.5.10 and the same
 problem remained.  Currently traffic is low so everything is load balanced
 evenly, but we expect another spike in a few hours and I expect the issue
 to return.


 Here's what haproxy-status looked like:




 Do I need to switch to maybe add a weight and tracking?  We have 12
 frontend appservers load balancing to 28.  All run haproxy and the app
 server software.

 Thanks!
 Alex

Re: rand(x) output limited to x/2

[Vivek Malik]

Hi Vincent,

Thanks for the patch. I was able to verify that the patch made the
distribution uniform over a large (300+) output from rand.

Regards,
Vivek

On Mon, Dec 8, 2014 at 4:45 PM, Vincent Bernat ber...@luffy.cx wrote:
  ❦  8 décembre 2014 23:20 +0100, Vincent Bernat ber...@luffy.cx :

 Assuming that RAND_MAX is always a power of two - 1, 32 could be
 replaced by a precomputed value of ffs(RAND_MAX+1)-1.

 ebtree defines a fls64() function which seems best suited (RAND_MAX+1
 could overflow). Here is a proposed patch for this:




 --
 Let the machine do the dirty work.
 - The Elements of Programming Style (Kernighan  Plauger)

Re: using HAProxy in front of SSO

[Vivek Malik]

Hi,

AFAIK, Haproxy doesn't have a subrequest feature.

However, there are other design solutions possible to make Haproxy as
SSO filter. I am using Haproxy is a similar fashion. I have coded my
SSO to notify Haproxy whenever a new SSO authentication session is
added. Haproxy adds that session id to the map. When a request comes
to Haproxy, it checks the map for the session id. If session id is not
present, haproxy is set to respond with 302 to the login page.

You can do the same with a stick_table too instead of a map. However,
you will need in_table(table) function which is only present in
1.6-devel as of now.

Regards,
Vivek


On Tue, Dec 9, 2014 at 6:54 PM, Patrick Kaeding
pkaed...@launchdarkly.com wrote:
 Hello

 I'm interested in using HAProxy as my external-facing proxy, in front
 of my applications. I want to implement an SSO application to handle
 authentication (similar to what is described here:
 http://dejanglozic.com/2014/10/07/sharing-micro-service-authentication-using-nginx-passport-and-redis/).

 Nginx has the ngx_http_auth_request_module
 (http://nginx.org/en/docs/http/ngx_http_auth_request_module.html),
 which looks like it would work well, but I am wondering if I can do
 this with HAProxy, and not need Nginx as a second layer in front of my
 applicaitons.

 Can HAProxy make subrequests to determine how to handle the incoming
 request? Are there any resources I should check out to help with this?

 Thanks!
 --
 Patrick Kaeding
 pkaed...@launchdarkly.com

rand(x) output limited to x/2

[Vivek Malik]

Hi,

I am using rand(x) in configuration to make some routing decisions. I
am basically load balancing between backends and using the following
configuration

use_backend bk_1 { rand(100) le 50 }
default_backend bk_2

However, I am not seeing any traffic going to bk_2 and all traffic
goes to bk_1. It seems that there is a bug in smp_fetch_rand function
around reduction.

I did some further testing by setting up a header using

http-request set-header X-RAND %[rand(200)]

and printing that header in a file. I am unable to see the random
value going above arg/2.

Here is my haproxy build information.

HA-Proxy version 1.5.9 2014/11/26
Copyright 2000-2014 Willy Tarreau w...@1wt.eu

Build options :
  TARGET  = linux2628
  CPU = generic
  CC  = gcc
  CFLAGS  = -O2 -g -fno-strict-aliasing
  OPTIONS = USE_ZLIB=1 USE_OPENSSL=1 USE_STATIC_PCRE=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200

Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.8
Compression algorithms supported : identity, deflate, gzip
Built with OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014
Running on OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 8.31 2012-07-06
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with transparent proxy support using: IP_TRANSPARENT
IPV6_TRANSPARENT IP_FREEBIND

Available polling systems :
  epoll : pref=300,  test result OK
   poll : pref=200,  test result OK
 select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Re: Can't get HAProxy to support Forward Secrecy FS

[Vivek Malik]

Are you putting in DH parameters in mycert.pem?

PFS depends on using DH algorithm to exchange and create a secret for
the connection.

openssl dhparam 2048  mycert.pem should add the DH parameters to the
cert file.

Regards,
Vivek

On Mon, Dec 8, 2014 at 4:44 PM, Sander Rijken san...@sanderrijken.nl wrote:
 System is Ubuntu 12.04 LTS server, with openssl 1.0.1 and haproxy 1.5.9

 OpenSSL version
 OpenSSL 1.0.1 14 Mar 2012


 I'm currently using the following, started with the suggested [stanzas][1]
 (formatted for readability, it is one long line in my config):

 bind 0.0.0.0:443 ssl crt mycert.pem no-tls-tickets ciphers \
 ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384: \

 ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA384: \

 ECDHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256: \
 AES128-SHA:AES256-SHA256:AES256-SHA no-sslv3

 [1]: https://gist.github.com/rnewson/8384304

 ssllabs.com indicates FS is not used. When I disable all algorithms except
 the ECDHE ones, I get SSL connection error (ERR_SSL_PROTOCOL_ERROR), so
 something on the system doesn't support FS.

 Any ideas?


 --
 Sander Rijken

map peering tcp vs unix socket call url before forwarding

[Vivek Malik]

Hi,

I have a few questions around setting up a customized haproxy farm. I
am combining all my questions in single email.

1) Map peering

Is it possible to peer maps across haproxy running on multiple
machines? Or store key, value pairs inside stick table (which supports
peering).

2) Backend of backends

I am using haproxy to load balance between multiple backends (A/B
testing). I am currently creating a new frontend for each backend in
the A/B/C test which listens on unix socket. Based on previous email
threads, I believe that listening on TCP socket gives better
performance than listening on unix sockets. So frontend listening on
443 forwards request to frontend listening on unix socket which then
forwards the request to actual backend servers.

Would listening on localhost tcp ports (instead of unix sockets) be
the recommended approach?

I am managing multiple A/B tests using same haproxy farm. So, do I
have to ensure that test X gets the same port across haproxy soft
restarts? I wouldn't want test 34 traffic to forward to test 35
because test 34 port got assigned to test 35 when config file was
rewritten using chef.

3) Call URL before forwarding

Is there any way to call a specific URL to validate the request and
then forward the request based on response from the previous call? I
am using this to see if user is logged in. If the user is not logged
in, I want to redirect the user to login screen. Does another reverse
proxy engine like Nginx have such feature?

Regards,
Vivek

Custom authentication integration

[Vivek Malik]

Hi,

I would like to set up a configuration where users are required to
authenticate via a custom authentication web app before navigating to
the content backend.

I was thinking on lines of haproxy server sending the request to the
auth server when a map entry is missing. When the user authenticates,
the authentication app will insert an entry into the haproxy map and
put a cookie in http response with same entry value.

Questions:
a) Is it possible to make entries expire in map?
b) Does this design look feasible?

Thanks for your input. I have used haproxy 1.4 quite a bit, but
haven't used 1.5/map feature/stick table yet.

Regards,
Vivek

Re: Stick on extracted regex value from header

[Vivek Malik]

Hi Baptise,

Thanks for the suggestion. The list of possible values is in 100s and I
would like to load balance on different machines while still keeping them
sticky. So, stick tables is the ideal solution. However, stick tables only
support pattern extractions which are limited to using full values of a
header or using url parameters.

Using the acls would mean

a) writing 100s of acls and adding more acls whenever a new possible value
is added.
b) defining a server statically for every possible value instead of letting
haproxy choose a server based on load balancing algorithm.

Is there a plan to add stick table pattern extraction which includes
extraction values from path or header using regular expressions?

Thanks,
Vivek


On Fri, Mar 1, 2013 at 12:54 AM, Baptiste bed...@gmail.com wrote:

 On Thu, Feb 28, 2013 at 7:28 PM, Vivek Malik vivek.ma...@gmail.com
 wrote:
  Hi,
 
  I see that it is possible to haproxy sticky on a header value, path,
 query
  parameter. It seems that haproxy does process the request URI to extract
  query parameters. Is is possible to use stick tables with stickyness on a
  value extracted from header?
 
  Eg: HTTP request includes header
 
  X-Custom-Authorization: 2-y89nfsjdhns
 
  Is is possible to make haproxy extract 2 i.e. (^(\d+)-.+) from the custom
  header and stick on the extracted value.
 
  Thanks,
  Vivek

 Hi Vivek,

 Better using the below acl:
 hdr_beg(X-Custom-Authorization) 2

 you can add as many values as you want.

 Baptiste

Re: HAProxy sometimes selects wrong acl

[Vivek Malik]

Oops. I read your email wrong. Re-reading your email. I see that you are
having problem with haproxy + long polling and websockets.

There was a discussion earlier in this thread on websockets breaking http
protocol (See http://marc.info/?l=haproxym=127803250504340w=2. Also see
http://stackoverflow.com/questions/4360221/haproxy-websocket-disconnection)

For Comet (long polling), haproxy might be closing the connection due to
server timeout.

I personally use websockets and comets by connecting client directly to the
backend server for websocket and comet connections bypassing the load
balancer all together. However, there might be better ways to do it which I
am not aware of.

Regards,
Vivek

On Sun, Oct 14, 2012 at 7:46 PM, Vivek Malik vivek.ma...@gmail.com wrote:

 I am not sure of how mikrokit could be having problems with
 http-server-close. http-server-close only affects the haproxy - backend
 connection, so it hidden from any front end.

 Having said that, I can think of one way to do http-server-close for
 specific conditions. It might not be the best method, but here it is.

 option http-server-close can be specified in backend. So, you can have a
 normal backend with http-server-close enabled and a mikrotik backend with
 http-server-close disabled. You can then use acl to choose the backend
 depending on http headers.

 Regards,
 Vivek

 On Sun, Oct 14, 2012 at 4:05 PM, Milan Babuskov 
 milan.babus...@gmail.comwrote:

 On Fri, Oct 12, 2012 at 4:51 PM, Vivek Malik vivek.ma...@gmail.com
 wrote:
  Yes, option http-server-close would make haproxy run acls for every
 request.
  It will also enable keep-alive between client (or proxy) and haproxy.

 Thanks, this fixed to problem for Mikrotik users, but now I have
 problems with Safari users who cannot establish websockets or
 xhr-polling connections anymore :(

 So, I reverted to old config and have a new question. Is it possible
 to activate http-server-close option only for Mikrotik users? I read
 the docs, but appears I can only access http headers withing acls for
 frontend selection?

 I would like to achieve something like:

 option http-server-close WHEN http headers CONTAIN Mikrotik

 Thanks,

 --
 Milan Babuskov
 http://www.guacosoft.com

Re: HAProxy sometimes selects wrong acl

[Vivek Malik]

You need to add option httpclose or option http-server-close so that
haproxy doesn't act in the tunnel mode (default behavior).

Regards,
Vivek

On Thu, Oct 11, 2012 at 5:33 PM, Milan Babuskov milan.babus...@gmail.comwrote:

 Hello,

 I'm using HAProxy 1.4.22. I have the following haproxy.conf file:

  haproxy.conf ---
 global
 maxconn 10
 daemon

 defaults
 mode http
 retries 1
 contimeout 8000
 clitimeout 12
 srvtimeout 12
 stats enable
 stats uri /haproxy-stats
 stats auth admin:
 option httpchk

 frontend http-in
 bind 16.9.13.39:80
 maxconn 10
 acl is_l1 hdr_end(host) -i l1.mydomain.com
 acl is_l2 hdr_end(host) -i l2.mydomain.com
 acl is_l3 hdr_end(host) -i l3.mydomain.com
 acl is_l0 hdr_end(host) -i mydomain.com
 use_backend lora1 if is_l1
 use_backend lora2 if is_l2
 use_backend lora3 if is_l3
 use_backend lora0 if is_l0
 default_backend lora0

 backend lora0
 balance roundrobin
 option forwardfor except 127.0.0.1  # stunnel already adds the header
 server s0 127.0.0.1:5000 check inter 6

 backend lora1
 balance source
 option forwardfor except 127.0.0.1  # stunnel already adds the header
 server s1 127.0.0.1:5001 check inter 6

 backend lora2
 balance source
 option forwardfor except 127.0.0.1  # stunnel already adds the header
 server s2 127.0.0.1:5002 check inter 6

 backend lora3
 balance source
 option forwardfor except 127.0.0.1  # stunnel already adds the header
 server s3 127.0.0.1:5003 check inter 6
  haproxy.conf end ---

 It all works fine. Except, for some 0.2% of the clients. Sometimes
 when the request comes for l1, l2 or l3. For example:

 http://l3.mydomain.com/something

 and HAProxy does not match the domain name for some reason and uses
 default backend instead. I have set up logging in my application and
 it reports that hostname on the receiving end is in fact
 l3.mydomain.com. Here are the headers that my application receives:

 --- headers captured by Node.js/Express application --
 host: 'l3.mydomain.com',
 'user-agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.4 (KHTML,
 like Gecko) Chrome/22.0.1229.94 Safari/537.4',
 accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
 referer: '
 http://mydomain.com/menu/2034414/e2e1abb5500ed51391d6351b1cf03695',
 'accept-encoding': 'gzip,deflate,sdch',
 'accept-language': 'en-US,en;q=0.8',
 'accept-charset': 'ISO-8859-1,utf-8;q=0.7,*;q=0.3',
 'x-proxy-id': '1407537728',
 'x-forwarded-for': '10.201.4.168',
 via: '1.1 10.201.255.254 (Mikrotik HttpProxy)'
 --

 I don't know what are the actual headers HAProxy receives from the
 client, but I assume this Mikrotik proxy is mangling it somehow and
 HAProxy fails to match the hostname. Is there some way to enable
 logging only for situation when none of the acl rules is matched and
 default backend is selected?

 Thanks,

 --
 Milan Babuskov
 guacosoft.com

Re: request per second statistics

[Vivek Malik]

I did something similar for graphite using bash and socat to read from
haproxy socket.

Script at http://pastebin.com/ccenegXb

Regards,
Vivek

On Tue, Oct 2, 2012 at 4:08 PM, Joel Krauska jkrau...@gmail.com wrote:

 I use something like this:
 (it has a few things that are specific to my implementation)

 #!/usr/bin/python
 import os
 import csv
 import urllib2
 import pprint
 import time

 datadir = '/home/rrdcollect/haproxy-stats/rrds'
 hostlist = ['a.network.com']
 haproxycreds = ['url','user','pass']
 key_values = ['scur', 'bin', 'bout']
 for host in hostlist:
print host

url = 'http://%s/%s;csv;norefresh' % (host, haproxycreds[0])
print url
pp = pprint.PrettyPrinter(indent=4)
passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
passman.add_password(None, url, haproxycreds[1], haproxycreds[2])
authhandler = urllib2.HTTPBasicAuthHandler(passman)
opener = urllib2.build_opener(authhandler)
urllib2.install_opener(opener)
f = urllib2.urlopen(url)

firstrow = True
e = dict()
for row_ind,row in enumerate(csv.reader(f)):
   if firstrow:
  mapping = row
  firstrow = False
   if FRONTEND in row or BACKEND in row:
  d = {}
  for ind,col_val in enumerate(row):
 col_name = mapping[ind]
 d[col_name]=col_val
 row_key = row[0],row_ind
 e[row_key]=d
for keys in key_values:
   for key_tuple,row_dict in e.iteritems():
  row_val = row_dict.get(keys,'')
  if keys == scur:
 pitem = CurrentSessionRate
  elif keys == bin:
 pitem = bpsIn
 row_val = int(row_val)*8
 row_val = str(row_val)
  elif keys == bout:
 pitem = bpsOut
 row_val = int(row_val)*8
 row_val = str(row_val)

  rrd = %s/haproxyStat_%s_%s_%s.rrd % (datadir,
 pitem, key_tuple[0], host)
  print rrd
  cmd = rrdtool update %s N:%s % (rrd, row_val)
  print cmd
  if not os.path.isfile(rrd):
 print 'RRD does not exist!'
 from time import time
 now = int(time())
 print now
 now-=300
 print now
 maxvalue =
 10
 if bps in pitem:
cmd = 'rrdtool create %s --start %s
 --step 300 DS:data:DERIVE:600:0:%s RRA:AVERAGE:0.5:1:9000
 RRA:AVERAGE:0.5:12:4000 RRA:AVERAGE:0.5:72:5000
 RRA:AVERAGE:0.5:288:2500' % (rrd, now, maxvalue)
 else:
cmd = 'rrdtool create %s --start %s
 --step 300 DS:data:COUNTER:600:0:%s RRA:AVERAGE:0.5:1:9000
 RRA:AVERAGE:0.5:12:4000 RRA:AVERAGE:0.5:72:5000
 RRA:AVERAGE:0.5:288:2500' % (rrd, now, maxvalue)
 print cmd
 os.system(cmd)
  os.system(cmd)



 On Tue, Oct 2, 2012 at 11:17 AM, Saul Waizer saulwai...@gmail.com wrote:
  Hello List,
 
  This is a general question, has anyone developed a method of polling
  statistics from HAproxy to determine the current requests per second
 rate? I
  am building a custom nagios plugin that will also work with graphite and
  cacti, i am trying to find the best approach to gather RPS statistics.
 Any
  suggestions?
 
  Thank You,
  Saul

Re: [ANNOUNCE] haproxy 1.4.21

[Vivek Malik]

A recommended upgrade for all production users. While we are not
(generally) affected by the bugs fixed in haproxy stable version. I
recommend updating haproxy.

I can update haproxy bin in puppet and can check it in (we distribute
haproxy binary via puppetmaster).

Aiman,

Please update puppetmaster when you see fit and also in general, please
ensure that puppet client is running on all machines.

Thanks,
Vivek

On Mon, May 21, 2012 at 1:43 AM, Willy Tarreau w...@1wt.eu wrote:

 Hi all,

 a number of old bugs were reported recently. Some of them are quite
 problematic because they can lead to crashes while parsing configuration
 or when starting up, which is even worse considering that startup scripts
 will generally not notice it.

 Among the bugs fixed in 1.4.21, we can enumerate :
  - risk of crash if using reqrep/rsprep and having tune.bufsize manually
configured larger than what was compiled in. The cause is the trash
buffer used for the replace was still static, and I believed this was
fixed months ago but only my mailbox had the fix! Thanks to Dmitry
Sivachenko for reporting this bug.

  - risk of crash when using header captures on a TCP frontend. This is a
configuration issue, and this situation is now correctly detected and
reported. Thanks to Olufemi Omojola for reporting this bug.

  - risk of crash when some servers are declared with checks in a farm which
does not use an LB algorithm (eg: option transparent or dispatch).
This happens when a server state is updated and reported to the non-
existing LB algorithm. Fortunately, this happens at start-up when
reporting the servers either up or down, but still it's after the fork
and too late for being easily recovered from by scripts. Thanks to David
Touzeau for reporting this bug.

  - balance source did not correctly hash IPv6 addresses, so IPv4
connections to IPv6 listeners would always get the same result. Thanks
to Alex Markham for reporting this bug.

  - the connect timeout was not properly reset upon connection
 establishment,
resulting in a retry if the timeout struck exactly at the same
 millisecond
the connect succeeded. The effect is that if a request was sent as part
 of
the connect hanshake, it is not available for resend during the retry
 and
a response timeout is reported for the server. Note that in practice,
 this
only happens with erroneous configurations. Thanks to Yehuda Sadeh for
reporting this bug.

  - the error captures were wrong if the buffer wrapped, which happens when
capturing incorrectly encoded chunked responses.

 I also backported Cyril's work on the stats page to allow POST params to be
 posted in any order, because I know there are people who script actions on
 this page.

 This release also includes doc cleanups from Cyril, Dmitry Sivachenko and
 Adrian Bridgett.

 Distro packagers will be happy to know that I added explicit checks to shut
 gcc warnings about unchecked write() return value in the debug code.

 While it's very likely that almost nobody is affected by the bugs above,
 troubleshooting them is annoying enough to justify an upgrade.

 Sources, Linux/x86 and Solaris/sparc binaries are at the usual location :

site index : http://haproxy.1wt.eu/
sources: http://haproxy.1wt.eu/download/1.4/src/
changelog  : http://haproxy.1wt.eu/download/1.4/src/CHANGELOG
binaries   : http://haproxy.1wt.eu/download/1.4/bin/

 Willy

Re: Is there any method to block malicious clients

[Vivek Malik]

Haproxy 1.5 has src_conn_rate which can be used for that. I personally
haven't used it. I just remember reading about it.

Vivek

On Tue, Mar 13, 2012 at 8:30 AM, Jerry Champlin 
j...@absolute-performance.com wrote:

 One way to do this is to find it in the logs with a script and then have
 that script apply a black hole rule to iptables.  As a matter of course, we
 use a similar approach to block rapid failed login attempts on servers with
 public facing ssh.  It works very well.

 -Jerry

 Jerry Champlin
 Absolute Performance Inc.
 Phone: 303-565-4401
 --
 Enabling businesses to deliver critical applications at lower cost and
 higher value to their customers.



 On Tue, Mar 13, 2012 at 2:57 AM, fred hu frederick...@gmail.com wrote:

 Hi, All

 We are using haproxy since 2009 for LB.

 Recently we encountered some malicious clients sending request on same
 URL with especially high rate ( 100r/s and lasting for some minutes)
 Is there any possibility to block such user while keep serving the normal
 clients? (Surly We have no idea on malicious users ip before (s)he attacks)
 I read the configuration manual and find we have
 fe_sess_rate/be_sess_rate ACLs. But it seems for all clients.

 So, my question here is : Can we find/block a malicious user based on his
 request rate?

 Thx!

 --
 *Fred Hu*
 *Best Regards*

Re: HTTP - HTTPS redirects

[Vivek Malik]

We have the same situation where we accept traffic for a long list of
domains. We solve the problem by adding the following for each domain

redirect prefix https://sub.domain.com if !is_https make_https {
hdr(host) sub.eqaldev.com }

The above solution only works if the list of the subdomains is finite and
can be iterated upon so that a line can be added for each subdomain. We do
reload the haproxy config every time a new subdomain is added.

Vivek

On Mon, Mar 5, 2012 at 3:18 PM, Finn Arne Gangstad fin...@gmail.com wrote:

 On Mon, Mar 5, 2012 at 7:03 PM, Richard Stanford rich...@kimbia.com
 wrote:
  [...]
 
  Our challenge is that our servers accept connections on a wildcard
 hostname
  ( *.domain.com ) and we need to maintain the host identity during the
  redirection, so hardcoding the host portion won't work in our case.
  Would
  somebody be able to help me out with the correct syntax, if its even
  possible?  Thanks!

 This cannot be done with haproxy. We use nginx for this, but apache
 obviously works too.

 - Finn Arne

Re: SSL Pass through and sticky session

[Vivek Malik]

You are running haproxy in a tcp mode since you are relaying SSL and
decrypting on the backend. Cookies can only be analyzed in HTTP mode. Not
sure how to do sticky sessions in tcp mode.

Vivek

On Mon, Nov 7, 2011 at 2:03 PM, Mir Islam mis...@mirislam.com wrote:

 Is it possible to utilize some sort of sticky session for incoming
 requests? SSL connections are terminated at the servers in the backend.
 Right now I can do source IP based balance. But then users behind a
 firewall/NAT will not get load balanced correctly. Instead, they all end up
 on same server. That is my main problem.


 Here is a portion of my config. I added the cookie param but I guess it
 will work with http only. Anyway, any help/pointer is appreciated.



 listen  ssl-relay 0.0.0.0:443
option  ssl-hello-chk
balance source
server  inst1 10.254.2.145:443 check inter 2000 fall 3
server  inst2 10.46.19.211:443 check inter 2000 fall 3

option  httpclose   # disable keep-alive
option  checkcache  # block response if set-cookie 
 cacheable

cookie HASERVERID inser

Re: Autoscaling in haproxy with persistence sessions

[vivek . malik]

If the solution is intended for traffic burst, Isn't it safe to assume that 
most clients will be new which appsession/cookie doesn't know about?

New clients will automatically be preferred to go to newly added servers as new 
servers will have least active connections.

I don't think any special change is required in practice to handle burst of new 
traffic from say a premium ad buy or email blast (along with using maxidle)

Vivek
Sent via BlackBerry from T-Mobile

-Original Message-
From: David Birdsong david.birds...@gmail.com
Date: Mon, 7 Nov 2011 12:17:53 
To: Erik Torlenerik.tor...@apicasystem.com
Cc: Vivek Malikvivek.ma...@gmail.com; 
haproxy@formilux.orghaproxy@formilux.org
Subject: Re: Autoscaling in haproxy with persistence sessions

This sounds like what balancing on a hashed value is intended for.
'hash-type consistent' will reduce the redistribution of keys when the
pool is expanded, and when nodes are removed, only the removed nodes
keys are redistributed.

On Mon, Nov 7, 2011 at 11:15 AM, Erik Torlen
erik.tor...@apicasystem.com wrote:
 Interesting. In this case we are expecting a lot of burst traffic during a
 very short period of time, 15-30min so I am not sure if we can rely on
 scaling in a more proactive way to send traffic to the new servers. I would
 be
 more comfortable if we could just clean the existing sessions and let them
 be spread over the new servers + existing servers.



 I had a look at stick-table and saw that it has methods to support being
 deleted/cleared through the socket interface. Is it possible to do something
 similar to clean appsessions? Or maybe store appsession in
 a stick-table and clear the session through socket command?



 /E



 From: Vivek Malik [mailto:vivek.ma...@gmail.com]
 Sent: den 7 november 2011 11:05
 To: Erik Torlen
 Cc: haproxy@formilux.org
 Subject: Re: Autoscaling in haproxy with persistence sessions



 I personally find it easier to use cookie instead of appsession. We use a
 similar pattern of adding a new server. Keeping a low maxidle (like 10
 minutes) helps us send traffic to new servers. Keeping maxidle helps us
 maintain session affinity where required (like progress bars for uploads)



 Vivek

 On Mon, Nov 7, 2011 at 1:32 PM, Erik Torlen erik.tor...@apicasystem.com
 wrote:

 Hi,

 We are currently having a system which runs haproxy in the amazon cloud. Our
 system is also using autoscaling of backendservers
 so when we reach a certain cpu usage during x min we will add more servers
 to the backend and update the haproxy config + reloading haproxy.

 This works good as we have it now.

 What we would like is to add persistence to the backend in order to use the
 caches on the backend servers more efficiently (a shared cache would have
 been
 better but is not the case now unfortunately).

 This makes the autoscaling a bit more complex because of the persistence.
 When scaling up new servers the client would still stay on the overloaded
 backend servers instead of start using the new ones.

 So I thought I would check with you if there is a way to clear persistence
 session used by appsession in a good way without effecting the traffic to
 servers?

 If we cleared all the persistence sessions we could let the client go into
 the new backend servers and have request-learn in appsession learn the
 cookie and set persistence to the existing and new servers for the client.

 Any ideas here?

 Cheers
 E

Re: haproxy and multi location failover

[vivek . malik]

DNS propagation can take a long time based on my experience. We have a similar 
problem where we host multiple identical setups in different EC2 availability 
zones. We have been thinking of having DNS entry with multiple A records for 
load distribution and failover. However, that doesn't solve the problem of OP.

Vivek
Sent via BlackBerry from T-Mobile

-Original Message-
From: Baptiste bed...@gmail.com
Date: Tue, 1 Nov 2011 18:17:25 
To: Senthil Naidusenthil.na...@gmail.com
Cc: Gene Jgh5...@gmail.com; haproxy@formilux.orghaproxy@formilux.org
Subject: Re: haproxy and multi location failover

There is not (yet) a GSLB or dyndns daemon available in opensource,
but a few DNS server could be used to emulate this feature.
- PowerDNS  + pipe backend
- unbound + python module

or yourself updating your DNS server to trigger a failover


Cheers


On Tue, Nov 1, 2011 at 6:10 PM, Senthil Naidu senthil.na...@gmail.com wrote:
 hi,

 we need to have a setup as follows



 site 1 site 2

   LB  (ip 1)   LB (ip 2)
    |   |
    |   |
  srv1  srv2  srv1 srv2

 site 1 is primary and site 2 is backup in case of site 1  LB's failure or
 failure of all the servers in site1 the website should work from backup
 location servers.

 Regards

 On Tue, Nov 1, 2011 at 10:31 PM, Gene J gh5...@gmail.com wrote:

 Please provide more detail about what you are hosting and what you want to
 achieve with multiple sites.

 -Eugene

 On Nov 1, 2011, at 9:58, Senthil Naidu senthil.na...@gmail.com wrote:

 Hi,

 thanks for the reply,  if the same needs to be done with dns do we need
 any external dns services our we can use our own ns1 and ns2 for the same.

 Regards


 On Tue, Nov 1, 2011 at 9:06 PM, Baptiste bed...@gmail.com wrote:

 Hi,

 Do you want to failover the Frontend or the Backend?
 If this is the frontend, you can do it through DNS or RHI (but you
 need your own AS).
 If this is the backend, you have nothing to do: adding your servers in
 the conf in a separated backend, using some ACL to take failover
 decision and you're done.

 cheers


 On Tue, Nov 1, 2011 at 2:25 PM, Senthil Naidu senthil.na...@gmail.com
 wrote:
  Hi,
 
  Is it possible to use haproxy in a active/passive failover scenario
  between
  multiple datacenters.
 
  Regards
 
 
 
 

Re: haproxy and multi location failover

[Vivek Malik]

May I ask what some of the acronyms in this email thread stand for

RHI -
AS -
GSLB -

Thanks,
Vivek

On Tue, Nov 1, 2011 at 2:26 PM, Baptiste bed...@gmail.com wrote:

 True :)
 Despite short TTLs, some client would take a long time to failover.
 But it's the only option unless you own your AS and you are able to
 route your traffic inside it.

 rgs


 On Tue, Nov 1, 2011 at 6:30 PM,  vivek.ma...@gmail.com wrote:
  DNS propagation can take a long time based on my experience. We have a
 similar problem where we host multiple identical setups in different EC2
 availability zones. We have been thinking of having DNS entry with multiple
 A records for load distribution and failover. However, that doesn't solve
 the problem of OP.
 
  Vivek
  Sent via BlackBerry from T-Mobile
 
  -Original Message-
  From: Baptiste bed...@gmail.com
  Date: Tue, 1 Nov 2011 18:17:25
  To: Senthil Naidusenthil.na...@gmail.com
  Cc: Gene Jgh5...@gmail.com; haproxy@formilux.orghaproxy@formilux.org
  Subject: Re: haproxy and multi location failover
 
  There is not (yet) a GSLB or dyndns daemon available in opensource,
  but a few DNS server could be used to emulate this feature.
  - PowerDNS  + pipe backend
  - unbound + python module
 
  or yourself updating your DNS server to trigger a failover
 
 
  Cheers
 
 
  On Tue, Nov 1, 2011 at 6:10 PM, Senthil Naidu senthil.na...@gmail.com
 wrote:
  hi,
 
  we need to have a setup as follows
 
 
 
  site 1 site 2
 
LB  (ip 1)   LB (ip 2)
 |   |
 |   |
   srv1  srv2  srv1 srv2
 
  site 1 is primary and site 2 is backup in case of site 1  LB's failure
 or
  failure of all the servers in site1 the website should work from backup
  location servers.
 
  Regards
 
  On Tue, Nov 1, 2011 at 10:31 PM, Gene J gh5...@gmail.com wrote:
 
  Please provide more detail about what you are hosting and what you
 want to
  achieve with multiple sites.
 
  -Eugene
 
  On Nov 1, 2011, at 9:58, Senthil Naidu senthil.na...@gmail.com
 wrote:
 
  Hi,
 
  thanks for the reply,  if the same needs to be done with dns do we need
  any external dns services our we can use our own ns1 and ns2 for the
 same.
 
  Regards
 
 
  On Tue, Nov 1, 2011 at 9:06 PM, Baptiste bed...@gmail.com wrote:
 
  Hi,
 
  Do you want to failover the Frontend or the Backend?
  If this is the frontend, you can do it through DNS or RHI (but you
  need your own AS).
  If this is the backend, you have nothing to do: adding your servers in
  the conf in a separated backend, using some ACL to take failover
  decision and you're done.
 
  cheers
 
 
  On Tue, Nov 1, 2011 at 2:25 PM, Senthil Naidu 
 senthil.na...@gmail.com
  wrote:
   Hi,
  
   Is it possible to use haproxy in a active/passive failover scenario
   between
   multiple datacenters.
  
   Regards
  
  
  
  
 
 
 
 
 

client side keep-alive (http-server-close vs httpclose)

[Vivek Malik]

We have been using haproxy in production for around 6 months while using
httpclose. We use functions like reqidel, reqadd to manipulate request
headers and use_backend to route a request to a specific backend.

We run websites which often have ajax calls and load javascripts and css
files from the server. Thinking about keep alive, I think it would be
desired to keep client side keep alive so that they can reuse connections to
load images, javascript, css and make ajax calls over it.

From a haproxy request processing and manipulating perspective, Is there a
difference between http-server-close and httpclose? Would
reqadd/reqidel/use_backend work on subsequent requests during client side
keep alive too?

I tried running some tests and I was able to reqadd/reqidel and use_backend
while using http-server-close, but I wanted to check with the group before
pushing the change to production.

Also, what's a good keep alive value for a web server. I was thinking around
10 seconds which would slow clients (include mobile) enough time to process
an html document and initiate requests for additional resources.

Thanks,
Vivek

Re: Graceful configuration changes?

[Vivek Malik]

haproxy -f /etc/haproxy.cfg -sf $(cat $PIDFILE) would do it

haproxy -h
HA-Proxy version 1.4.18 2011/09/16
Copyright 2000-2011 Willy Tarreau w...@1wt.eu

Usage : haproxy [-f cfgfile]* [ -vdVD ] [ -n maxconn ] [ -N maxpconn ]
[ -p pidfile ] [ -m max megs ]
-v displays version ; -vv shows known build options.
-d enters debug mode ; -db only disables background mode.
-V enters verbose mode (disables quiet mode)
-D goes daemon
-q quiet mode : don't display messages
-c check mode : only check config files and exit
-n sets the maximum total # of connections (4)
-m limits the usable amount of memory (in MB)
-N sets the default, per-proxy maximum # of connections (4)
-p writes pids of all children to this file
-de disables epoll() usage even when available
-ds disables speculative epoll() usage even when available
-dp disables poll() usage even when available
-dS disables splice usage (broken on old kernels)
-sf/-st [pid ]* finishes/terminates old pids. Must be last
arguments.


Also see http://haproxy.1wt.eu/download/1.2/doc/haproxy-en.txt

Vivek


On Wed, Oct 19, 2011 at 9:00 PM, John Singleton jsing...@gmail.com wrote:

 Hi Kevin,

 I believe you can just send it a -HUP signal. eg: kill -HUP pid

 Best,
 JLS


 On Wed, Oct 19, 2011 at 4:37 PM, Kevin Lindsay ke...@trakker.ca wrote:

 Can HA Proxy make graceful configuration changes? Or does the entire proxy
 need to be restarted leaving space for downtime?

 Thanks,
 Kevin

Re: redirect prefix, use variable host

[Vivek Malik]

I take it back. The hack doesn't seem to work. Even after changing the host
in the request, prefix / is redirecting to /+uri and doesn't include the
full host. So, its doing a relative redirect which doesn't change the
protocol from http to https and vice-versa.

Stiil looking for a solution.

Thanks,
Vivek

On Wed, Apr 20, 2011 at 10:43 PM, Vivek Malik vivek.ma...@gmail.com wrote:

 I found a hacky way to do it from mailing list archives

 http://www.mail-archive.com/haproxy@formilux.org/msg03323.html

 Regards,
 Vivek

 On Wed, Apr 20, 2011 at 10:09 PM, Vivek Malik vivek.ma...@gmail.comwrote:

 Hi,

 I am trying to issue a redirect via haproxy and want to reuse the host
 variable. Here is the configuration

 stunnel running on port 443 accepting connections for *.domain.com(unlimited 
 subdomain cert) forwarding to haproxy on port 81
 haproxy running on port 80 (for http for *.domain.com) and port 81 (for
 https via stunnel)

 frontend https
bind 127.0.0.1:81
acl allow_https if path /login /signup /forgotPassword
redirect prefix http://sub.domain.com if !allow_https

 Stunnel is using an unlimited subdomain SSL certificate which allows for
 https connection to *.domain.com. I want to redirect to
 http://sub.domain.com but I can't put the value of the sub without using
 a variable. I tried looking around but can't find out how to use the
 variable host inside the redirect prefix directive.

 Essentially, the config should be able to handle redirect for all
 subdomains

 https://foo.domain.com/something = http://foo.domain.com/something
 https://bar.domain.com/something = http://bar.domain.com/something

 Please help.

 Thanks,
 Vivek

Re: redirect prefix, use variable host

[Vivek Malik]

I found a hacky way to do it from mailing list archives

http://www.mail-archive.com/haproxy@formilux.org/msg03323.html

Regards,
Vivek

On Wed, Apr 20, 2011 at 10:09 PM, Vivek Malik vivek.ma...@gmail.com wrote:

 Hi,

 I am trying to issue a redirect via haproxy and want to reuse the host
 variable. Here is the configuration

 stunnel running on port 443 accepting connections for *.domain.com(unlimited 
 subdomain cert) forwarding to haproxy on port 81
 haproxy running on port 80 (for http for *.domain.com) and port 81 (for
 https via stunnel)

 frontend https
bind 127.0.0.1:81
acl allow_https if path /login /signup /forgotPassword
redirect prefix http://sub.domain.com if !allow_https

 Stunnel is using an unlimited subdomain SSL certificate which allows for
 https connection to *.domain.com. I want to redirect to
 http://sub.domain.com but I can't put the value of the sub without using a
 variable. I tried looking around but can't find out how to use the variable
 host inside the redirect prefix directive.

 Essentially, the config should be able to handle redirect for all
 subdomains

 https://foo.domain.com/something = http://foo.domain.com/something
 https://bar.domain.com/something = http://bar.domain.com/something

 Please help.

 Thanks,
 Vivek

Re: using haproxy for https

[vivek . malik]

Haproxy can't do ssl ... SSL is encrypted from client to Server .. So haproxy 
can't analyze the requests and add headers or look at headers.

You should be able to run haproxy in tcp mode and balance based on source but 
you can't load balance based on cookie or header info unless ssl decryption 
happens before the traffic reaches haproxy.

Vivek
-Original Message-
From: Joseph Hardeman jwharde...@gmail.com
Date: Sun, 10 Apr 2011 13:57:24 
To: haproxy@formilux.org
Subject: Re: using haproxy for https

Hi Guys

The problem is that this is for a customer who is running IIS and already
has all their certs built for IIS, I don't know if the IIS cert would work
with stunnel.

I tried the following configuration which I had found and they said it was
working for them, but I am getting SSL to long errors:

#listen cust1_443
#maxconn 32000
#bind0.0.0.0:443
#mode http
#cookie SERVERID insert indirect nocache
##cookie SERVERID rewrite nocache
#timeout client 70s
#timeout server 70s
#timeout connect 30s
#balance source
#reqadd X-Forwarded-Proto:\ https
#reqadd SSL-TERMINATION:\ ON
#server IIS1-443 192.168.0.206:443 cookie iis1ssl check inter 5000
fall 3 rise 1 maxconn 30
##server IIS2-443 192.168.0.207:443 cookie iis2ssl check inter 5000
fall 3 rise 1 maxconn 30
#option abortonclose
#option httpclose
#option forwardfor
#retries 3
#option redispatch
#log global
#option httplog
#option ssl-hello-chk
#option dontlognull


With the second IIS server commented out, they are able to serve 1 of their
largest customer with their SSL site, but I want to be able to load balance
the requests and at least pin each visitor to IIS server they are sent to.

listen  cust1_443
mode tcp
bind 0.0.0.0:443
option ssl-hello-chk
balance roundrobin
server IIS1-443 192.168.0.206:443 check inter 5000 fall 3 rise 1
maxconn 300
#   server IIS2-443 192.168.0.207:443 check inter 5000 fall 3 rise 1
maxconn 300
timeout client 70s
timeout server 70s
timeout connect 30s

Any ideas or thoughts on this?

Thanks

JOe


On Sun, Apr 10, 2011 at 10:26 AM, Brian Carpio bcar...@broadhop.com wrote:

 You probably need to ask that question on the stunnel mailing list.


 Sent from my iPhone

 On Apr 10, 2011, at 8:20 AM, German Gutierrez germ...@olx.com wrote:

  BTW, will this patch ever go upstream? Why stunnel does not have this
 already?
 
  On Sat, Apr 9, 2011 at 11:43 PM, Vivek Malik vivek.ma...@gmail.com
 wrote:
  Joe,
  You need to run as many stunnel instances as number of SSL certificates.
 If
  the sites share SSL certificate, then one stunnel instance will do.
  I run stunnel 4.32 with patch from
 http://haproxy.1wt.eu/download/patches/
  on port 443 and forward it to port 81 on the same machine which is bound
 to
  haproxy.
  My stunnel config looks like
  cert = /etc/stunnel.pem
  sslVersion = all
  chroot = /var/lib/stunnel/
  setuid = stunnel
  setgid = stunnel
  pid = /stunnel.pid
  socket = l:TCP_NODELAY=1
  socket = r:TCP_NODELAY=1
  [https]
  accept  = 443
  connect = 127.0.0.1:81
  TIMEOUTclose = 0
  xforwardedfor = yes
  Note that xforwardedfor option only works after the patch is installed.
  My
  haproxy config looks like
  frontend http
  bind 0.0.0.0:80
  reqidel ^X-Forwarded-Proto:.*
  reqadd X-Forwarded-Proto:\ HTTP
  option forwardfor
  frontend https
  bind 127.0.0.1:81
  reqidel ^X-Forwarded-Proto:.*
  reqadd X-Forwarded-Proto:\ HTTPS
  Note that I am passing a X-Forwarded-Proto to underlying application so
 that
  it can logic specific to https calls.
  Vivek
  On Sat, Apr 9, 2011 at 4:00 PM, Ben Timby bti...@gmail.com wrote:
 
  On Sat, Apr 9, 2011 at 2:07 PM, Joseph Hardeman jwharde...@gmail.com
  wrote:
  Hi Guys,
 
  I was wondering if someone has a good example I could use for proxying
  https
  traffic.  We are trying to proxy multiple sites that use https and I
 was
  hoping for a way to see how to proxy that traffic between multiple IIS
  servers without having to setup many different backend sections.  The
  way
  the sites are setup they use a couple of cookies but mostly session
  variables to track the user as they do their thing.  Either I need to
 be
  able to pin the user to a single server using the mode tcp function
 when
  they come in or be able to use some form of mode http that doesn't
 break
  the
  SSL function.
 
  This morning around 5am, I got one site running with only 1 backend
  using
  tcp but I really need to be able to load balance it between multiple
  servers.
 
  Joe, haproxy itself does not do SSL. That said, you can set up an SSL
  server in front of it. Myself, I use stunnel. Stunnel strips the SSL
  and forwards the traffic to haproxy. I have many instances of stunnel
  (one per

Re: using haproxy for https

[Vivek Malik]

Joe,

You need to run as many stunnel instances as number of SSL certificates. If
the sites share SSL certificate, then one stunnel instance will do.

I run stunnel 4.32 with patch from
http://haproxy.1wt.eu/download/patches/on port 443 and forward it to
port 81 on the same machine which is bound to
haproxy.

My stunnel config looks like

cert = /etc/stunnel.pem
sslVersion = all
chroot = /var/lib/stunnel/
setuid = stunnel
setgid = stunnel
pid = /stunnel.pid
socket = l:TCP_NODELAY=1
socket = r:TCP_NODELAY=1
[https]
accept  = 443
connect = 127.0.0.1:81
TIMEOUTclose = 0
xforwardedfor = yes

Note that xforwardedfor option only works after the patch is installed.  My
haproxy config looks like

frontend http
bind 0.0.0.0:80
reqidel ^X-Forwarded-Proto:.*
reqadd X-Forwarded-Proto:\ HTTP
option forwardfor

frontend https
bind 127.0.0.1:81
reqidel ^X-Forwarded-Proto:.*
reqadd X-Forwarded-Proto:\ HTTPS

Note that I am passing a X-Forwarded-Proto to underlying application so that
it can logic specific to https calls.

Vivek

On Sat, Apr 9, 2011 at 4:00 PM, Ben Timby bti...@gmail.com wrote:

 On Sat, Apr 9, 2011 at 2:07 PM, Joseph Hardeman jwharde...@gmail.com
 wrote:
  Hi Guys,
 
  I was wondering if someone has a good example I could use for proxying
 https
  traffic.  We are trying to proxy multiple sites that use https and I was
  hoping for a way to see how to proxy that traffic between multiple IIS
  servers without having to setup many different backend sections.  The way
  the sites are setup they use a couple of cookies but mostly session
  variables to track the user as they do their thing.  Either I need to be
  able to pin the user to a single server using the mode tcp function when
  they come in or be able to use some form of mode http that doesn't break
 the
  SSL function.
 
  This morning around 5am, I got one site running with only 1 backend using
  tcp but I really need to be able to load balance it between multiple
  servers.

 Joe, haproxy itself does not do SSL. That said, you can set up an SSL
 server in front of it. Myself, I use stunnel. Stunnel strips the SSL
 and forwards the traffic to haproxy. I have many instances of stunnel
 (one per cert/ip) which all feed a single haproxy http listener.

 http://www.stunnel.org/

 You could also use another server like nginx, apache etc. to strip the
 SSL. However, I find stunnel well suited as all it does is SSL and it
 is fast and efficient at it (similar to how haproxy does proxyinig
 very well).

Possible bug - request forwarding to wrong backend

[Vivek Malik]

Hi,

I have been struggling with this bug for a little while and wanted to report
it and see if you can either help me or fix this bug. Here is a sample
config file which shows the bug

*global*
*log 127.0.0.1 local0*
*user nobody*
*group nogroup*
*daemon*
*pidfile /var/run/haproxy.pid*
*
*
*defaults*
*log global*
*mode http*
*option httplog*
*
*
*timeout client 10s*
*timeout connect 5s*
*timeout http-request 10s*
*timeout server 60s*
*
*
*frontend http*
*bind 0.0.0.0:80 http://0.0.0.0/*
* acl path_group_a path_beg /login /logout /admin /secure /js/ /css/*
* *
* user_backend back_a if path_group_a*
* default_backend back_b*
* *
*backend back_a*
*server a1 10.20.203.165:80 http://10.20.203.165/*
*backend back_b*
*server b1 10.20.122.68:80 http://10.20.122.68/*
*
*
*
*
Basically, the config is just distributing http connections between 2
servers based on the path. However, I am constantly seeing haproxy sending
traffic to the wrong backend. The backend back_b in the example above is
responsible for all pages except a few like /login, /logout, /admin, /secure
etc.

The bug can be reproduced by
a) User goes to home page. Haproxy forwards the request to back_b.
b) User clicks the login link (/login which is supposed to be sent to
back_a).
c) User receives response from back_b instead of back_a

If the user clicks on a link quickly the request ends up going to back_b
instead of back_a as expected based on ACL. Moreover, haproxy doesn't log
the request and its routing to back_b. Whenever haproxy logs the request, it
is sending to the right backend. The wrong backend logs do show them
receiving requests from ip running haproxy. So, it has to be haproxy
forwarding those requests.

I have tried running haproxy in debug mode too but am unable to see the
requests there too. Somehow the request is getting routed to wrong backend
and there is no log generated for that bug in info mode or debug mode.

Please help ! I am trying to set up a production infrastructure and solving
this problem would be a great help.

Thanks,
Vivek
310-754-5861

Re: Possible bug - request forwarding to wrong backend

[Vivek Malik]

Hi Cyril,

Thanks for the quick response. I tried adding option foreclose and option
http-pretend-keepalive and it seems to fix the situation.

I learned a few things about http keep-alive today ...

Thanks :)

Vivek

On Mon, Apr 4, 2011 at 2:33 AM, Cyril Bonté cyril.bo...@free.fr wrote:

  Hi Vivek,


 Le lundi 04 avril 2011 11:21:55, Vivek Malik a écrit :

  Hi,

 

  I have been struggling with this bug for a little while and wanted to
 report

  it and see if you can either help me or fix this bug. Here is a sample

  config file which shows the bug

  (...)

  Basically, the config is just distributing http connections between 2

  servers based on the path. However, I am constantly seeing haproxy
 sending

  traffic to the wrong backend. The backend back_b in the example above is

  responsible for all pages except a few like /login, /logout, /admin,
 /secure

  etc.

 

  The bug can be reproduced by

  a) User goes to home page. Haproxy forwards the request to back_b.

  b) User clicks the login link (/login which is supposed to be sent to

  back_a).

  c) User receives response from back_b instead of back_a

 

  If the user clicks on a link quickly the request ends up going to back_b

  instead of back_a as expected based on ACL. Moreover, haproxy doesn't log

  the request and its routing to back_b. Whenever haproxy logs the request,
 it

  is sending to the right backend. The wrong backend logs do show them

  receiving requests from ip running haproxy. So, it has to be haproxy

  forwarding those requests.


 Your sample config file uses tunnel mode, which means it can only analyze
 the first request of a connection. Everything after the headers of this
 first request is considered as data.

 The issue you encounter is probably due to HTTP KeepAlive.


 You should try to add option httpclose, or better option
 http-server-close (which enables client HTTP keepalive and allows to
 analyze each request of a connection).

 With option http-server-close, you can also have a look at timeout
 http-keep-alive and option http-pretend-keepalive.


 --

 Cyril Bonté