talking http to a https port

2018-08-15 Thread Angelo Höngens
Hey guys,

I'm building a solution where I am ssl-offloading some old plain http
applications.

The http ports are listening on 8001, 8002, 8003, etc. I'm using haproxy
1.8.12 to listen on ports 18001, 18002, 18003, etc. using ssl. Everything
works fine, as expected.

I am trying to make stuff a bit more fool-proof for stupid users, and
testing stuff like talking http to https ports, etc. These things are not
supposed to happen, but users sometimes do stupid stuff like that.

When I talk http to a https port, I would expect something like an error
message 'ERROR 400: You are talking http to a https port', or something
like that. This makes it clear to users what they are doing wrong. (Apache
does this.)

However, I get no reply whatsoever. Not even a status code, haproxy just
closes the connection, and curl just says: "* Empty reply from server".
Haproxy logs a nice "SSL handshake failure" error in it's logs, as expected.

I don't know if haproxy was designed to just keep quiet, but I would love
to get a nice error message back. Is this something configurable?

Angelo.


RE: Haproxy failover: DNS RR vs Virtual IP (heartbeat, keepalived)

2011-01-06 Thread Angelo Höngens
(sorry for top posting, damn Outlook)

Just to second Willy's story, this is how a lot of people do it, including us. 
We use pacemaker for high availability, and dnr rr for loadbalancing. 

For example we have a 4-node cluster running varnish and haproxy. In this case 
I have 4 virtual ipv4-addresses and 4 virtual ipv6 addresses on the cluster. We 
use pacemaker to keep the virtual ip's up, and we use dns round-robin to 
balance the load. We get nice equal load balancing this way, and if a node is 
down (or I want to do maintenance), the vip's move to other nodes, and they 
take the extra load. 

-- 

 
With kind regards,
 
 
Angelo Höngens
 
Systems Administrator
 
--
NetMatch
tourism internet software solutions
 
Ringbaan Oost 2b
5013 CA Tilburg
T: +31 (0)13 5811088
F: +31 (0)13 5821239
 
mailto:a.hong...@netmatch.nl
http://www.netmatch.nl
--

 -Original Message-
 From: Willy Tarreau [mailto:w...@1wt.eu]
 Sent: donderdag 6 januari 2011 8:20
 To: David
 Cc: haproxy@formilux.org
 Subject: Re: Haproxy failover: DNS RR vs Virtual IP (heartbeat,
 keepalived)
 
 Hi David,
 
 On Thu, Jan 06, 2011 at 02:19:23PM +0900, David wrote:
  Hi,
 
  Let's say I have an architecture where a couple of servers are put
  behind a haproxy instance for load balancing, to serve content at
  www.example.com. For reliability/availability purpose, I want to have
  more than one haproxy instance to ensure continuous servicing when
 one
  haproxy fails:
 
LB1   LB2
 | |
  
  || |
  Server1   Server2   Server3
 
  The issue is how to distribute the load across both load balancers.
 I
  am aware of at least two solutions:
 
- DNS Round Robin: www.example.com is resolved to both LB1 and
 LB2's
  IP. If e.g. LB1 crashes, clients will then look at the next entry,
 LB2
  in this case
- High Availability IP (heartbeat, keepalive) between both load
  balancers. Only one load balancer is proxying all the requests at a
 time
  (I assume one load balancer has enough power to serve all our
 traffic).
 
  I have been asked to implement the DNS RR method, but from what I
 have
  read, method 2 is the one most commonly used. What are the pros/cons
 of
  each ?
 
 The first one is just pure theory. You may want to test it by yourself
 to conclude that it simply does not work at all. Most clients will see
 a connection error or timeout, and few of them will be able to perform
 a retry on the other address but after some delay which will cause some
 unpleasant experience. Also, most often the browser does not perform a
 new lookup if the first one has already worked. That means that until
 the browser is closed, the visitor will remain bound to the same IP.
 
 Then you might think that it's enough to update the DNS entries upon
 failure, but that does not propagate quickly, as there are caches
 everywhere. To give you an idea, the haproxy ML and site were moved to
 a new server one month ago, and we're still receiving a few requests a
 minute on the old server. In general you can count on 1-5% of the
 visitors
 to cache an entry more than one week. This is not a problem for a
 disaster
 recovery, but it certainly is for a server failover because that means
 you
 cannot put it offline at all.
 
 High availability has the big advantage of always exposing a working
 service for the same IP address, so it's a *lot* more reliable and
 transparent to users. There are two common ways to provide HA under
 Linux :
   - heartbeat
   - keepalived
 
 The first one is more suited to data servers, as it ensures that no
 more
 than one node is up at a time. This is critical when you share file
 systems.
 The second one is more suited to stateless servers such as proxies and
 load
 balancers, as it ensures that no less than one node is up at a time.
 Sadly
 people generally confuse them and sometimes use keepalived for NFS
 servers
 or use heartbeat with haproxy...
 
 High availability presents a small inconvenient though : the backup
 node
 is never used so you don't really know if it works well, and there is a
 big
 temptation not to update it as often as the master node. This is also
 an
 advantage in that it allows you to validate your new configs on it
 before
 loading them on the master node. If you want to use both LBs at the
 same
 time, the solution is to have two crossed VIPs on your LBs and use DNS
 RR
 to ensure that both are used. When one LB fails, the VIP moves to the
 other
 one.
 
 If you stick to the following principles, you should never encounter
 issues :
   - DNS = load balancing, no availability at all
   - HA = availability, no load balancing at all.
   = use DNS to announce always available IP addresses
 
 Cheers,
 Willy
 




RE: Haproxy failover: DNS RR vs Virtual IP (heartbeat, keepalived)

2011-01-06 Thread Angelo Höngens
Round robin is not the same as random.

If a host name has 4 A records, then most DNS servers (if not all) will return 
it round-robin. So first a.a.a.1, then a.a.a.2, then a.a.a.1, then a.a.a.2, 
etc.. Of course there are multiple dns servers involved and thousands of 
clients over the world, but in the end we almost always get a perfect 
25-25-25-25 balancing.

-- 

 
With kind regards,
 
 
Angelo Höngens
 
Systems Administrator
 
--
NetMatch
tourism internet software solutions
 
Ringbaan Oost 2b
5013 CA Tilburg
T: +31 (0)13 5811088
F: +31 (0)13 5821239
 
mailto:a.hong...@netmatch.nl
http://www.netmatch.nl
--

 -Original Message-
 From: David [mailto:da...@silveregg.co.jp]
 Sent: donderdag 6 januari 2011 9:45
 To: haproxy@formilux.org
 Subject: Re: Haproxy failover: DNS RR vs Virtual IP (heartbeat,
 keepalived)
 
 On 01/06/2011 05:01 PM, Angelo Höngens wrote:
  (sorry for top posting, damn Outlook)
 
  Just to second Willy's story, this is how a lot of people do it,
 including us. We use pacemaker for high availability, and dnr rr for
 loadbalancing.
 
  For example we have a 4-node cluster running varnish and haproxy. In
 this case I have 4 virtual ipv4-addresses and 4 virtual ipv6 addresses
 on the cluster. We use pacemaker to keep the virtual ip's up, and we
 use dns round-robin to balance the load. We get nice equal load
 balancing this way, and if a node is down (or I want to do
 maintenance), the vip's move to other nodes, and they take the extra
 load.
 
 
 Thanks for the information. Both Willy and you refer to DNS RR as a
 load
 balancing solution, but I don't really understand that point: if
 caching, etc... means hostname-load balancer resolution is random, the
 load balancing will likely be very unbalanced, no ?
 
 cheers,
 
 David




Re: Haproxy failover: DNS RR vs Virtual IP (heartbeat, keepalived)

2011-01-06 Thread Angelo Höngens
On 6-1-2011 9:59, David wrote:
 If a host name has 4 A records, then most DNS servers (if not all)
 will return it round-robin. So first a.a.a.1, then a.a.a.2, then
 a.a.a.1, then a.a.a.2, etc.. Of course there are multiple dns servers
 involved and thousands of clients over the world, but in the end we
 almost always get a perfect 25-25-25-25 balancing.
 
 Hm, interesting, I have seen reports of different behaviors. What kind
 of configurations may interfere with the ideal, near uniform balancing ?

Dunno.. what I do know is that the more users you get, the better the
balancing :)

Here you see a request graph of one of my clusters:

http://files.hongens.nl/2011/01/06/varnishrequests.png

As you see the distribution is not perfect, but for me it's enough.


-- 


With kind regards,


Angelo Höngens
systems administrator

MCSE on Windows 2003
MCSE on Windows 2000
MS Small Business Specialist
--
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
+31 (0)13 5811088
+31 (0)13 5821239

a.hong...@netmatch.nl
www.netmatch.nl
--





RE: Get real source IP

2010-11-16 Thread Angelo Höngens
Read Willy's response from earlier today, he showed the command.. (I use 
another product in front of haproxy that removes the header in my case).

-- 

 
With kind regards,
 
 
Angelo Höngens
 
Systems Administrator
 
--
NetMatch
tourism internet software solutions
 
Ringbaan Oost 2b
5013 CA Tilburg
T: +31 (0)13 5811088
F: +31 (0)13 5821239
 
mailto:a.hong...@netmatch.nl
http://www.netmatch.nl
--


 -Original Message-
 From: Maxime Ducharme [mailto:m...@techboom.com]
 Sent: dinsdag 16 november 2010 16:52
 To: Angelo Höngens
 Cc: 'Graeme Donaldson'; 'haproxy@formilux.org'
 Subject: RE: Get real source IP
 
 
 Hi Angelo
 
 Thanks for reply
 
 Can you enlighten me on how to remove a specific header ?  example :
 
 we will use X-MySrc-Ip as header name, I would like to ensure people
 cannot spoof it. Can we configure haproxy to remove any X-MySrc-Ip
 headers before adding the source IP ?
 
 --
 Maxime Ducharme
 Systems Architect
 
 
 On Mon, 2010-11-15 at 21:06 +, Angelo Höngens wrote:
  Or you could remove the client’s xff header, and always use your own.
  Then you are sure you can trust your own xff header, and the client
  can’t bypass.
 
 
 
  --
 
 
 
  With kind regards,
 
 
 
  Angelo Höngens
 
  Systems Administrator
 
  --
 
  NetMatch
 
  tourism internet software solutions
 
  Ringbaan Oost 2b
 
  5013 CA Tilburg
 
  T: +31 (0)13 5811088
 
  F: +31 (0)13 5821239
 
  mailto:a.hong...@netmatch.nl
 
  http://www.netmatch.nl
 
  --
 
 
 
  From: Graeme Donaldson [mailto:gra...@donaldson.za.net]
  Sent: maandag 15 november 2010 20:17
  To: Maxime Ducharme
  Cc: haproxy@formilux.org
  Subject: Re: Get real source IP
 
 
 
 
  On 15 November 2010 21:09, Maxime Ducharme m...@techboom.com wrote:
 
  Hi guys
 
  We are looking for a way to get real source IP that is connecting to
  our
  web services.
 
  We currently use option forwardfor, but some people are using this
  to
  bypass our checks.
 
  Is there other way to send real IP to our web servers ?
 
 
 
 
  Another way to do this is to use HAproxy in transparent proxy mode. I
  have not used it personally, but unless I'm mistaken it functions
 more
  like a NAT/routing device instead of a proxy.
 
 
 
 
 
  Here's a short howto if you'd like to try it
  out: http://blog.loadbalancer.org/configure-haproxy-with-tproxy-
 kernel-for-full-transparent-proxy/
 
 
 
 
 
  Regards,
 
 
  Graeme.
 
 



RE: Get real source IP

2010-11-15 Thread Angelo Höngens
Or you could remove the client's xff header, and always use your own. Then you 
are sure you can trust your own xff header, and the client can't bypass.

--

With kind regards,

Angelo Höngens
Systems Administrator
--
NetMatch
tourism internet software solutions
Ringbaan Oost 2b
5013 CA Tilburg
T: +31 (0)13 5811088
F: +31 (0)13 5821239
mailto:a.hong...@netmatch.nl
http://www.netmatch.nl
--

From: Graeme Donaldson [mailto:gra...@donaldson.za.net]
Sent: maandag 15 november 2010 20:17
To: Maxime Ducharme
Cc: haproxy@formilux.org
Subject: Re: Get real source IP

On 15 November 2010 21:09, Maxime Ducharme 
m...@techboom.commailto:m...@techboom.com wrote:
Hi guys

We are looking for a way to get real source IP that is connecting to our
web services.

We currently use option forwardfor, but some people are using this to
bypass our checks.

Is there other way to send real IP to our web servers ?

Another way to do this is to use HAproxy in transparent proxy mode. I have not 
used it personally, but unless I'm mistaken it functions more like a 
NAT/routing device instead of a proxy.

Here's a short howto if you'd like to try it out: 
http://blog.loadbalancer.org/configure-haproxy-with-tproxy-kernel-for-full-transparent-proxy/

Regards,
Graeme.


RE: haproxy server ipv6 support

2010-11-05 Thread Angelo Höngens
I also want to say I'd really like to see full end-to-end support over ipv6. 

It's not a top priority for us, and you are the one setting your priorities of 
course. All I'm saying is you might want to reconsider those priorities now, 
with the current ipv4 netblock exhaustion day estimated in June 2011.

We will see more and more people that require ipv6 in their setup. I also have 
plans for future setups to be ipv6-only internally. From client to 
balancer/cache the traffic can be ipv4/ipv6, but there's no need for ipv4 
anymore on the backends, since everything's ipv6-enabled there (and keeping 
ipv4 around makes things more complex).

(for the interested, here are some more of my thoughts on ipv6 and where we are 
now: http://ipv6.netmatch.nl)

I don't need a reaction, just think about it ;)

-- 

 
With kind regards,
 
 
Angelo Höngens
 
Systems Administrator
 
--
NetMatch
tourism internet software solutions
 
Ringbaan Oost 2b
5013 CA Tilburg
T: +31 (0)13 5811088
F: +31 (0)13 5821239
 
mailto:a.hong...@netmatch.nl
http://www.netmatch.nl
--

 -Original Message-
 From: Willy Tarreau [mailto:w...@1wt.eu]
 Sent: vrijdag 5 november 2010 5:51
 To: Ramesh Natarajan
 Cc: haproxy@formilux.org
 Subject: Re: haproxy server ipv6 support
 
 On Thu, Nov 04, 2010 at 09:48:31PM -0700, Ramesh Natarajan wrote:
  Thanks for the quick response. We currently have a setup where a
 bunch of
  backend servers that are listening only on a ipv6 address.
  Are there any plans to consider implementing this feature in haproxy
 any time?
 
 Yes, there were plans but at a very low priority. It is very possible
 that someone will one day provide a patch to enable that and then I'll
 mergeit. I might also implement it on a raindy day when I have nothing
 else to do :-)
 
 But I can't suggest you a date nor a version.
 
 Regards,
 Willy
 




RE: Ipaddr Problems

2010-11-03 Thread Angelo Höngens

Or you could use pacemaker. We've been using it for a while now, and it seems 
to work great! (we needed ipv6 support, so needed something else on Linux).

I've published a page on my blog about it. Just the simple basics, getting a 
cluster with a highly-available ip running in 5 minutes:

http://blog.hongens.nl/guides/setting-up-a-pacemaker-cluster-on-centosrhel/

-- 

 
With kind regards,
 
 
Angelo Höngens
 
Systems Administrator
 
--
NetMatch
tourism internet software solutions
 
Ringbaan Oost 2b
5013 CA Tilburg
T: +31 (0)13 5811088
F: +31 (0)13 5821239
 
mailto:a.hong...@netmatch.nl
http://www.netmatch.nl
--

 -Original Message-
 From: Willy Tarreau [mailto:w...@1wt.eu]
 Sent: woensdag 3 november 2010 7:24
 To: Hank A. Paulson
 Cc: haproxy@formilux.org
 Subject: Re: Ipaddr Problems
 
 On Tue, Nov 02, 2010 at 12:15:27PM -0700, Hank A. Paulson wrote:
  If you are trying to failover only an IP address(es) and haproxy -
 do
  yourself a huge favor and just use keepalived. It is fast and
 painless to
  set up and maintain.
 
  http://www.keepalived.org/
 
 I agree too much with you !
 
 As I like to repeat it, the huge difference between heartbeat and
 keepalived
 is that heartbeat ensures that a resource is present *at most* at one
 place,
 while keepalived ensures that it is present *at least* at one place.
 The
 former is suited for sharing file systems. The later is suited for
 sharing
 stateless IP-based services like haproxy.
 
 Willy
 




RE: rpms for rhel5?

2010-11-01 Thread Angelo Höngens
Great, I'd really like to see 1.4.9 in the rhel5 epel repo :)

-- 

 
With kind regards,
 
 
Angelo Höngens
 
Systems Administrator
 
--
NetMatch
tourism internet software solutions
 
Ringbaan Oost 2b
5013 CA Tilburg
T: +31 (0)13 5811088
F: +31 (0)13 5821239
 
mailto:a.hong...@netmatch.nl
http://www.netmatch.nl
--


 -Original Message-
 From: Jeremy Hinegardner [mailto:jer...@hinegardner.org]
 Sent: maandag 1 november 2010 17:03
 To: Willy Tarreau
 Cc: Angelo Höngens; 'haproxy@formilux.org'
 Subject: Re: rpms for rhel5?
 
 On Sun, Oct 24, 2010 at 09:44:18PM +0200, Willy Tarreau wrote:
  On Sun, Oct 24, 2010 at 02:05:38PM +, Angelo H?ngens wrote:
   Hey, does anyone know a place I can find rhel5 rpm's for the latest
 1.4 releases? The popular repositories all seem to have only 1.3..
 
  I'm not aware of any, and since the original package was 1.3, it
 seems
  logical that updates for the same package remain in the same branch.
 
  However, 1.4.8 is packaged for RHEL6. Maybe you can simply perform an
  rpmbuild on the .src ?
 
 
 As the maintainer of haproxy in EPEL, I would be happy to update
 haproxy to the
 1.4.x in EPEL if 1.4 is backward compatible with the 1.3 configuration
 files.
 
 It looks like I haven't updated Fedora to haproxy 1.4, I will make sure
 that F14
 is on haproxy 1.4 soon.
 
 enjoy,
 
 -jeremy
 
 --
 ===
 =
  Jeremy Hinegardner  jer...@hinegardner.org




RE: VM benchmarks

2010-10-28 Thread Angelo Höngens
I'm wondering what the difference would be between the standard slow e1000 
virtual network card and the fast paravirtualized vmxnet3 virtual network card. 
In theory, the latter one should be much, much faster.. 

-- 

 
With kind regards,
 
 
Angelo Höngens
 
Systems Administrator
 
--
NetMatch
tourism internet software solutions
 
Ringbaan Oost 2b
5013 CA Tilburg
T: +31 (0)13 5811088
F: +31 (0)13 5821239
 
mailto:a.hong...@netmatch.nl
http://www.netmatch.nl
--


 -Original Message-
 From: Les Stroud [mailto:l...@lesstroud.com]
 Sent: woensdag 27 oktober 2010 21:55
 To: Ariel
 Cc: haproxy
 Subject: Re: VM benchmarks
 
 Check out this thread I had earlier in the month on the same topic:
 http://www.formilux.org/archives/haproxy/1010/3910.html
 
 Bottom line: vmware will slow down your upper level transaction limit
 by a significant amount (like an order of maginitude).  The software
 drivers underneath the network stack and the system stack add enough
 overhead to reduce your maximum transaction ceiling to around 6000
 trans/sec on haproxy (this is without a backend constraint).  On a
 hardware device, I am seeing much higher numbers (50k).
 
 LES
 
 
 On Oct 26, 2010, at 10:38 AM, Ariel wrote:
 
  Does anyone know of studies done comparing haproxy on dedicated
 hardware vs virtual machine?  Or perhaps some virtual machine specific
 considerations?
  -a
 




Re: rpms for rhel5?

2010-10-25 Thread Angelo Höngens
On 24-10-2010 21:44, Willy Tarreau wrote:
 On Sun, Oct 24, 2010 at 02:05:38PM +, Angelo Höngens wrote:
 Hey, does anyone know a place I can find rhel5 rpm's for the latest
 1.4 releases? The popular repositories all seem to have only 1.3..
 
 
 I'm not aware of any, and since the original package was 1.3, it
 seems logical that updates for the same package remain in the same
 branch.
 
 However, 1.4.8 is packaged for RHEL6. Maybe you can simply perform
 an rpmbuild on the .src ?

That works like a charm! I've put my own built rpm here, in case someone
else needs a quick fix:

http://files.netmatch.nl/RPMS/haproxy-1.4.8-1.x86_64.rpm

(although of course no one should trust binaries untrusted people
precompile)

-- 


With kind regards,


Angelo Höngens
systems administrator

MCSE on Windows 2003
MCSE on Windows 2000
MS Small Business Specialist
--
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
+31 (0)13 5811088
+31 (0)13 5821239

a.hong...@netmatch.nl
www.netmatch.nl
--





rpms for rhel5?

2010-10-24 Thread Angelo Höngens
Hey, does anyone know a place I can find rhel5 rpm's for the latest 1.4 
releases? The popular repositories all seem to have only 1.3.. 

I know I can compile from source, but I'm a lazy sysadmin, and I really like 
the ease of prebuilt packages. It's easier to install, remove, update, includes 
the right init script for my distro, etc.

-- 

 
With kind regards,
 
 
Angelo Höngens
 
Systems Administrator
 
--
NetMatch
tourism internet software solutions
 
Ringbaan Oost 2b
5013 CA Tilburg
T: +31 (0)13 5811088
F: +31 (0)13 5821239
 
mailto:a.hong...@netmatch.nl
http://www.netmatch.nl
--




RE: HAProxy Stunnel end-to-end SSL

2010-10-20 Thread Angelo Höngens
(oops, sent to Ryan instead of list)

Just out of interest: why not use plain http to the backends? 

We are making the move in our company to use all of our balancers as ssl 
offloaders, and just use plain http on the backends. The sites still need to 
know if a page was requested securely or not, so we send them the 
'front-end-https' header. Our asp.net applications can then see if a request 
came over http or https. Keeps the server configs quite simple, never an ssl 
certificate on a backend, and the expensive backends don't have to waste cpu 
power on encrypting and decrypting.

-- 

 
With kind regards,
 
 
Angelo Höngens
 
Systems Administrator
 
--
NetMatch
tourism internet software solutions
 
Ringbaan Oost 2b
5013 CA Tilburg
T: +31 (0)13 5811088
F: +31 (0)13 5821239
 
mailto:a.hong...@netmatch.nl
http://www.netmatch.nl
--

From: Clark, Ryan [mailto:ryan.cl...@xerox.com] 
Sent: woensdag 20 oktober 2010 21:02
To: Mike Hoffs
Cc: haproxy@formilux.org
Subject: RE: HAProxy Stunnel end-to-end SSL

I actually got it to work by using TCP mode. This might help other users to 
look at this config using stunnel and haproxy. Not sure how this is working at 
all, but it does. 

HAPROXY CONFIG:

global
   log 127.0.0.1 local0
   maxconn 4096
   uid 99
   gid 99
   daemon

defaults
   mode tcp
   log global
   option tcplog
   option httpclose
   retries 3
  maxconn 2000
   contimeout  5
   clitimeout  50
   srvtimeout  50


frontend LB1 *:443
   acl XSM-acl url_sub -i XeroxServicesManager
   acl XSP-acl url_sub -i XSP
   acl FMP-acl url_sub -i FMP
   use_backend XSM if XSM-acl
   use_backend XSP if XSP-acl
   use_backend XSP if FMP-acl
   default_backend MPSAPI

backend XSM
   option ssl-hello-chk
   balance roundrobin
   server ROCPRDXSM1 10.0.5.155:443 check
   server ROCPRDXSM2 10.0.5.156:443 check

backend XSP
   option ssl-hello-chk
   balance roundrobin
   server ROCPRDXSP1 10.0.5.19:443 check
   server ROCPRDXSP2 10.0.5.91:443 check

backend MPSAPI
   option ssl-hello-chk
   balance roundrobin
   server ROCPRDXDMC 10.0.5.158:443 check
   server ROCPRDCMPS 10.0.5.185:443 check

STUNNEL:

cert=/etc/certs/OFFICEB2.pem
;setuid = nobody
;setgid = nogroup

pid = /etc/stunnel/stunnel.pid
debug = 3
output = /etc/stunnel/stunnel.log

socket=l:TCP_NODELAY=1
socket=r:TCP_NODELAY=1

[https]
accept=10.0.5.161:443
connect=10.0.5.161:8080
TIMEOUTclose=0
xforwardedfor=yes



From: Mike Hoffs [mailto:m.ho...@mijn-sleutel.com] 
Sent: Wednesday, October 20, 2010 2:11 PM
To: Clark, Ryan
Subject: RE: HAProxy Stunnel end-to-end SSL

Hi Ryan,

Note offside mailinglist, last days there was someone with simular situation;


http://www.formilux.org/archives/haproxy/1010/3922.html
http://www.formilux.org/archives/haproxy/1010/date.html

Met een vriendelijke groet,   


Mike Hoffs

Mijn-Sleutel
Peperstraat 33
6678 AL Oosterhout
Tel: +31 (0)24 8200208 tijdens kantoor uren (09:00 - 17:00)
Mail: m.ho...@mijn-sleutel.com
Website: http://www.mijn-sleutel.com

Van: Clark, Ryan [mailto:ryan.cl...@xerox.com] 
Verzonden: woensdag 20 oktober 2010 20:00
Aan: Mike Hoffs; haproxy@formilux.org
Onderwerp: RE: HAProxy Stunnel end-to-end SSL

Yes I have, even with the option ssl-hello-chk enabled.

From: Mike Hoffs [mailto:m.ho...@mijn-sleutel.com] 
Sent: Wednesday, October 20, 2010 1:56 PM
To: Clark, Ryan; haproxy@formilux.org
Subject: RE: HAProxy Stunnel end-to-end SSL

Have u tried mode tcp ?


Met een vriendelijke groet,   


Mike Hoffs




RE: List of sites using haproxy ?

2010-07-28 Thread Angelo Höngens
We use it for www.zoover.nl, www.weeronline.nl, www.snp.nl, etc. (in 
combination with either squid or varnish caches though).

-- 

 
With kind regards,
 
 
Angelo Höngens
 
Systems Administrator
 
--
NetMatch
tourism internet software solutions
 
Ringbaan Oost 2b
5013 CA Tilburg
T: +31 (0)13 5811088
F: +31 (0)13 5821239
 
mailto:a.hong...@netmatch.nl
http://www.netmatch.nl
--


 -Original Message-
 From: Alexandre Snarskii [mailto:s...@snar.spb.ru]
 Sent: woensdag 28 juli 2010 11:40
 To: haproxy@formilux.org
 Cc: Alexandre Snarskii
 Subject: List of sites using haproxy ?
 
 
 Hi!
 
 Sorry that I'm starting my subscription with such a silly question,
 but is there any kind of list of projects/sites using haproxy as
 load balancer ? (offline responses are welcome too)
 
 Why I need it - one of our web-services performs much better using
 haproxy maxconn/queue feature (it protects database from spikes),
 but to implement it on production I need to get approval from
 management.
 And management says No, unless it's confirmed that this software is
 really used widely, and, as haproxy operates transparently it use is
 not shown in netcraft web server survey like in case of nginx..
 




iptables performance impact

2010-07-13 Thread Angelo Höngens
Hey, does anyone have an idea how iptables impacts network performance? (on 
CentOS 5.5 x64 for example).

I've got haproxy running on quite some FreeBSD machines for quite a while now, 
and I'm very happy with it. We have quite some different setups (directly on 
the net, behind cisco firewall in dmz, with host firewall, without, etc). Now 
we're slowly moving from FreeBSD to CentOS, and by default iptables is enabled.

On our FreeBSD machines that are directly connected to the net, we have a 
public interface with services only listening on port 80, and an internal 
interface for stats access, ssh and snmp. But we have some new machines on 
which we only want to use a single public interface. We'd use iptables to allow 
only trusted ip's to connect to management services.

What are your real-life experiences? Do you have iptables enabled on your 
balancers? Normally I would do stresstests, but somehow my stresstests never 
simulate real-world behavior with a mix of tens of thousands of slow and fast 
clients, etc. 

By the way, some of our balancers do  100 Mbit and  2000 req/s by the way.

If anyone has any best practices concerning this subject, I'd be glad to hear 
it as well.

-- 

 
With kind regards,
 
 
Angelo Höngens
 
Systems Administrator
 
--
NetMatch
tourism internet software solutions
 
Ringbaan Oost 2b
5013 CA Tilburg
T: +31 (0)13 5811088
F: +31 (0)13 5821239
 
mailto:a.hong...@netmatch.nl
http://www.netmatch.nl
--





Re: Sharing machine or separate machine?

2010-07-06 Thread Angelo Höngens
On 6-7-2010 10:32, Christian Jensen wrote:
  Hi,
 
 I am setting up a new datacenter and would love to get an opinion...
 
 We have 3 options:
 1. Build a firewall machine separate from the load balancer machine
 2. Share a machine and have a firewall and haproxy on the same box
 3. Virtualize everything (VMWare, Xen, KVM)
 
 Please suggest you best choice for firewall if you want - we can use
 anything. Also, if you have any decent experience with any hypervisor,
 please weigh in there too.
 
 Thanks!
 Christian


Please suggest a new car for me. I have three options: a pick-up truck,
a car with a trailer behind it, or a lorry truck. They can all be used
for transporting cargo, and probably do a good job, but I can't make you
any suggestions, since perhaps you do other work than I do. :-)

I can tell you what we do, based on the work we do..

About firewalls: we mainly use cisco firewalls everywhere (they're also
good for setting up a site-to-site vpn from your office to your
datacenter). We have haproxy, varnish and squid machines behind them.

For some high-volume projects we have some balancers attached directly
to the net. These balancers have at least 2 network cards, and the
'public' interface only has port 80 open. SSH and other services only
listen on the inside interface. In this case you don't really need a
firewall to close ports.

-- 


With kind regards,


Angelo Höngens
systems administrator

MCSE on Windows 2003
MCSE on Windows 2000
MS Small Business Specialist
--
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
+31 (0)13 5811088
+31 (0)13 5821239

a.hong...@netmatch.nl
www.netmatch.nl
--





Re: Sharing machine or separate machine?

2010-07-06 Thread Angelo Höngens
On 6-7-2010 18:01, Christian Jensen wrote:
  Thanks for the feedback. I do understand that it is pretty vague.
 
 If you have the machine directly on the net and all ports off, is the
 only reason to use a cisco to get the VPN or are there other benefits? I
 come from the M$ world where we used ISA server and I understand the
 positives there but also the downsides too.

The most important reason for me is control. If your server is
compromized and you lose control (I actually had a windows server where
I watched a hacker move my mouse around), how will you control the
server or limit the business impact? It's nice to have a small appliance
where you can control what's going in and out, even if your servers are
compromized.

 
 With ISA server I got:
 * URL Routing
 * NAT
 * VPN
 * Logging
 * Load Balancing
 * SSL offloading
 and more. I know this forum is not designed specifically for talking
 about Cisco products but does anyone know what the best box for the best
 price to emulate the above features? Is it simply a matter of mixing in
 other Open Source projects and boxes?

We use cisco asa's for NAT, routing (ipv6) and VPN, and I use my
balancers for the rest. The balancers run a combination of haproxy,
varnish, squid, apache, whatever fits my requirements.

-- 


With kind regards,


Angelo Höngens
systems administrator

MCSE on Windows 2003
MCSE on Windows 2000
MS Small Business Specialist
--
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
+31 (0)13 5811088
+31 (0)13 5821239

a.hong...@netmatch.nl
www.netmatch.nl
--





Re: Varnish vs HAProxy vs Nginx vs Squid vs Other

2010-07-06 Thread Angelo Höngens
On 6-7-2010 18:07, Christian Jensen wrote:
  Hi guys,
 
 I am biasing towards HAProxy but as the comparison shopper that I am, I
 am looking to make sure that is the right choice.
 
 Does anyone have a URL to a feature comparison chart to compare a bunch
 of different Load Balancers/Proxies?
 
 Sounds like a great wikipedia article to me.
 
 Thanks!
 Christian
 

Nope, but it's like comparing apples to pears. A load balancer, a
caching proxy and a modular web server are quire different products,
although they do have overlap in their functionality.

We use haproxy for balancing, because in my opinion it's best at
balancing. We use squid for balancing and ssl offloading, and varnish
for high-volume caching.

see diagram for inspiration:
http://files.hongens.nl/2010/05/21/2010_balancers_diagram.pdf

-- 


With kind regards,


Angelo Höngens
systems administrator

MCSE on Windows 2003
MCSE on Windows 2000
MS Small Business Specialist
--
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
+31 (0)13 5811088
+31 (0)13 5821239

a.hong...@netmatch.nl
www.netmatch.nl
--





Re: Varnish vs HAProxy vs Nginx vs Squid vs Other

2010-07-06 Thread Angelo Höngens
On 6-7-2010 20:28, Angelo Höngens wrote:
 We use haproxy for balancing, because in my opinion it's best at
 balancing. We use squid for balancing and ssl offloading, and varnish
 for high-volume caching.

reply-to-self: I meant squid/varnish for *caching*



-- 


With kind regards,


Angelo Höngens
systems administrator

MCSE on Windows 2003
MCSE on Windows 2000
MS Small Business Specialist
--
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
+31 (0)13 5811088
+31 (0)13 5821239

a.hong...@netmatch.nl
www.netmatch.nl
--





Re: question about transparancy

2010-04-23 Thread Angelo Höngens
On 22-4-2010 20:28, Michiel van Es wrote:
 Yes
 That is the default smtp failover setup but I want to balance the load  
 via a load balancer setup
 Mx records can not balance load

If you have 2 mx records with the same priority, your load should be
balanced..

Or you could have a single mx record pointing to a hostname which has 2
A records.. DNS round robin will take care of the balancing.

That is why there are almost no smtp balancers, because it is not
needed. In the 1980's they already designed smtp for balancing and
failover. For other protocols this was not so easy, that's why people
wrote http balancers :)


-- 


With kind regards,


Angelo Höngens
systems administrator

MCSE on Windows 2003
MCSE on Windows 2000
MS Small Business Specialist
--
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
+31 (0)13 5811088
+31 (0)13 5821239

a.hong...@netmatch.nl
www.netmatch.nl
--





Re: question about transparancy

2010-04-23 Thread Angelo Höngens
On 23-4-2010 11:19, Michiel van Es wrote:
 
 
 Angelo Höngens wrote:
 On 22-4-2010 20:28, Michiel van Es wrote:
 Yes
 That is the default smtp failover setup but I want to balance the load
 via a load balancer setup
 Mx records can not balance load

 If you have 2 mx records with the same priority, your load should be
 balanced..

 Or you could have a single mx record pointing to a hostname which has 2
 A records.. DNS round robin will take care of the balancing.

 That is why there are almost no smtp balancers, because it is not
 needed. In the 1980's they already designed smtp for balancing and
 failover. For other protocols this was not so easy, that's why people
 wrote http balancers :)

 Yes I understand, but what about settings features as weight or doe 
 advanced load balancing?


You can't do advanced balancing, true..

If you *must* have weight, you could go for the host records approach.
Make 1 MX record pointing to mx-in.example.com, and create three host
records: mx-in - x.x.x.1, mx-in - x.x.x.1, mx-in - x.x.x.2. This way,
server 1 gets around 66% of the sessions, and server 2 gets around 33%
of the sessions.


 What is one of the mailservers are broken and you want to take it offline.
 With a normal TTL in dns it can take 1 or 2 days before other 
 mailservers know it should not send a mail to that server and use the other.
 I like load balancers because they can let you decide how traffic must flow.


No problem if you use the MX way, just take the server offline, no need
to change dns.. Remote mail servers will just try one mail server, and
if it's down, they will use the other, failover is built into the way
smtp and dns work together.


I'm not saying you should not do what you are doing. If you really want
to use your own balancer, and you feel better doing that, then by all
means please do. What's I'm saying is that people have been balancing
smtp servers for 30 years using the ways they though of in the 80's, and
since that works for most organisations, it might work for you. KISS.

Don't look blindly at the tools you're using, but choose the tools you
need based on the goal you're trying to reach. Ah, who am I kidding, I'm
just an IT-nerd wanting to play with cool balancers as well..


-- 


With kind regards,


Angelo Höngens
systems administrator

MCSE on Windows 2003
MCSE on Windows 2000
MS Small Business Specialist
--
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
+31 (0)13 5811088
+31 (0)13 5821239

a.hong...@netmatch.nl
www.netmatch.nl
--





Re: acls and httpclose

2010-04-23 Thread Angelo Höngens
On 21-4-2010 23:54, Willy Tarreau wrote:
 I want to use haproxy for content switching on a large scale (lot of
 acls, lot of backends), but with httpclose on haproxy uses 25% cpu,
 without httpclose haproxy uses 5% cpu. So I'd rather not use httpclose
 if I don't have to..

 Also looks ok, since if you use httpclose haproxy got more work, so
 cpu also got more work.
 
 In fact it's not much more work for haproxy, but for the system, doing
 a connect is more expensive than a send of one packet. However, if you
 observe that large differences, I conclude that you're transfering very
 small objects so that the connect/close overhead becomes predominant.
 
 My observations are that http-server-close is about twice as fast as
 httpclose, so you could save about half of the CPU usage here.
 
 Willy
 

Trying to wrap my head around this.. I read the configuration guide, and
let me see if I understand correctly: when I use the http-server-close
option, connections between haproxy and the backends or not persistent,
but for each request, a new connection will be established.

But this adds the overhead of connection establishment, and should
increase cpu usage, not descrease it, right? What am I missing? (Keep in
mind I'm only talking about the haproxy machine, don't care about the
backend server.)

The http-close option does about the same thing, but it is about
connection between the clients and haproxy.

To me the backend-side and the client-side are two seperate things,
which have little to do with eachother. Still, you say
http-server-close is about twice as fast as httpclose. Am I not
interpreting things right?



-- 


With kind regards,


Angelo Höngens
systems administrator

MCSE on Windows 2003
MCSE on Windows 2000
MS Small Business Specialist
--
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
+31 (0)13 5811088
+31 (0)13 5821239

a.hong...@netmatch.nl
www.netmatch.nl
--





acls and httpclose

2010-04-21 Thread Angelo Höngens

Hey, I read somewhere on the list that when you use keepalives, only the
first request in the connection is matched to an acl, and then the other
requests in the connection are not evaluated.

I noticed this behavior as well. As an experiment I set up a large
config, where I select one out of 325 backends, based on one out of 8000
host headers. I noticed that only the first request in a connection is
matched to a backend, and the rest follows to the same backend, even
though the host header is different. With the httpclose option,
everything works as it should.

My question is: is this behavior by design, or is this a work-in-progress?

I want to use haproxy for content switching on a large scale (lot of
acls, lot of backends), but with httpclose on haproxy uses 25% cpu,
without httpclose haproxy uses 5% cpu. So I'd rather not use httpclose
if I don't have to..

-- 


With kind regards,


Angelo Höngens
systems administrator

MCSE on Windows 2003
MCSE on Windows 2000
MS Small Business Specialist
--
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
+31 (0)13 5811088
+31 (0)13 5821239

a.hong...@netmatch.nl
www.netmatch.nl
--





Re: haproxy 1.4.1 port crashes

2010-03-17 Thread Angelo Höngens
On 17-3-2010 16:10, Willy Tarreau wrote:
 Hello,
 
 On Wed, Mar 17, 2010 at 09:59:41AM +0100, Pawel Jaskorzynski wrote:
 Hello,
 I would like to report instability of the recent stable branch port of 
 haproxy-1.4.1. After some time of processing TCP requests in 
 round-robin, haproxy dies with broken pipe in debug. Nothing else gets 
 reported.
 My systsem is i386 FreeBSD 8.0-STABLE #1: Mon Mar 15, GENERIC kernel.
 The port has been cvsup'ed on the 16th of March.
 Previously operated haproxy-1.2.X for long time with no problems. 
 Currently reverted to haproxy-1.2.18, works as expected.
 
 I already got one report of a crash on FreeBSD. The fact that you see
 a broken pipe lets me think that MSG_NOSIGNAL is defined, causing the
 sigpipe not to be caught, but it does not have the expected effect.
 Could you please remove the #if/#endif around signal(SIGPIPE, SIG_IGN)
 in src/haproxy.c and try again ?
 
 Thanks,
 Willy
 

Me too..

I upgraded haproxy from 1.3.2x to 1.4.1 on one of my production
balancers this morning from the ports (yes, I was feeling lucky), and
noticed after a while haproxy was stopped. Nothing in any logs (no debug
settings), it just stopped after a few minutes. I'm running FreeBSD
7.2-RELEASE amd64 with a custom kernel (carp compiled in). Installed
1.3.22 from haproxy-devel, that works fine..

No time to dig deeper now, but if you need any help let me know, perhaps
I can do some tests this week if needed.
-- 


With kind regards,


Angelo Höngens
systems administrator

MCSE on Windows 2003
MCSE on Windows 2000
MS Small Business Specialist
--
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
+31 (0)13 5811088
+31 (0)13 5821239

a.hong...@netmatch.nl
www.netmatch.nl
--





Re: FreeBSD Ports: bumping haproxy from v1.2.18 - v1.4.x

2010-02-26 Thread Angelo Höngens
On 26-2-2010 16:42, Ross West wrote:
 
 Opening up a bit of discussion:
 
 For those Freebsd port users out there, I'm looking to submit updates
 for the haproxy port to take it from it's current v1.2.18 to the new
 v1.4.x tree - Leapfrogging the v1.3.x tree (which is part of the
 haproxy-devel port).
 
 Note: I'm _not_ looking to change the haproxy-devel port, which is
 currently part of the v1.3.x tree (v1.3.22 as of writing), and I
 believe is the port that most (all?) people are actually using.
 
 Obviously sometime in the future haproxy-devel should be changed to
 reference the snapshot or rc/dev builds that might be unstable, but
 that's not what I'm touching.
 
 Couple of benefits that I see of doing it this way:
 
 - Current systems running haproxy-devel port are untouched.
 
 - Less problems than pushing haproxy-devel to v1.4, and haproxy to
 v1.3, causing issues with config migrations for ports and software.
 
 - This'll eventually bring haproxy[-devel] back into line with the
 ports mentality of the main port being considered the active/stable
 port, with any sub ports being special cases.
 
 Main problems I see:
 
 - People running the current haproxy port (ie: v1.2.18) will have a
 big version bump to deal with.
 
 
 Any thoughts/complaints/etc?
 

I don't have a problem with your approach..

However, the way I think it should go in the ideal situation, is that
the haproxy port should contain the latest and greatest stable release
(1.4.x), and the haproxy-devel port should go to the latest experimental
snapshot.. If you think keeping a 1.3.x tree alive is usefull (which I
do), create a port haproxy13 for that..

Sure, you would bump the haproxy version up from 1.2 to 1.4, but people
who upgrade their ports should know to be careful around version upgrades..

As an example: If you switch from Squid 2.x to Squid 3.x your squid
won't start anymore if you have the acl 'ALL' defined in your config..
You get an error, you google it, and it turns out in 3.x, the acl is
already in the system, and hence you cannot define it again in your
config. I'm fine with that, as long as the errors are clear ;)


-- 


With kind regards,


Angelo Höngens
systems administrator

MCSE on Windows 2003
MCSE on Windows 2000
MS Small Business Specialist
--
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
+31 (0)13 5811088
+31 (0)13 5821239

a.hong...@netmatch.nl
www.netmatch.nl
--





Re: Perfect sysctl

2009-12-30 Thread Angelo Höngens
On 30-12-2009 14:04, Marcus Herou wrote:
 Hi Willy, thanks for your answer it got filtered, that's why I missed it
 for two weeks.
 
 Let's start with describing the service.
 
 We are hosting javascripts of the sizes up to 20K and serve flash and
 image banners as well which of course are larger. That is basically it..
 Ad Serving.
 
 On the LB's we have about 2MByte/s per LB  = 2x2MByte/s = 4MByte/s
 ~30MBit/s at peak, that is not the issue.
 
 I've created a little script which parse the active connections from
 the HAProxy stat interface and plots it into Cacti, it peaks at 100
 (2x100) connections per machine which is very little in your world I guess.
 
 I've attached a plot of tcp-connections as well. Nothing fancy there
 either besides that the number of TIME_WAIT sockets are in the 1
 range (log scale)
 
 Here's the problem:
 
 Everyother day I receive alarms from Pingdom that the service is not
 available and if I watch the syslog I get at about the same timings
 hints about possible SYN flood. At the same timings we receive emails
 from sites using us that our service is damn slow.
 
 What I feel is that we get hickups on the LB's somehow and that
 requests get queued. If I count the number of rows in the access logs on
 the machines behind the LB it decreases at the same timings and with the
 same factor on each machine (perhaps 10-20%) leading me to think that
 the narrow point is not on the backend side.


Maybe interesting, maybe not: I had some problems like this as well and
in my case I think this was caused by the limited number of outgoing
ports from my proxy machines..

I don't use connection keep-alives, and I think my balancers were
reusing ports faster than the backend windows machines could handle or
something like that.. Anyway, after I changed my FreeBSD's available
outgoing ports range, all problems were solved again.

Here's my sysctl for my FreeBSD 7.2 machines, but as Willy said, this
might not work for everyone.

kern.maxfiles=65535
kern.maxfilesperproc=32767
kern.ipc.maxsockbuf=16777216
kern.ipc.somaxconn=32768
net.inet.tcp.rfc1323=1
net.inet.tcp.sendbuf_max=16777216
net.inet.tcp.recvbuf_max=16777216
net.inet.tcp.sendbuf_auto=1
net.inet.tcp.sendbuf_inc=16384
net.inet.tcp.recvbuf_auto=1
net.inet.tcp.recvbuf_inc=524288
net.inet.tcp.inflight.enable=0
net.inet.tcp.hostcache.expire=1
net.inet.ip.portrange.first=1024
net.inet.ip.portrange.last=65535
net.inet.ip.portrange.hifirst=49152
net.inet.ip.portrange.hilast=65535

@Willy or someone else, feel free to comment on these settings if you
see something strange.

-- 


With kind regards,


Angelo Höngens
systems administrator

MCSE on Windows 2003
MCSE on Windows 2000
MS Small Business Specialist
--
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
+31 (0)13 5811088
+31 (0)13 5821239

a.hong...@netmatch.nl
www.netmatch.nl
--





RE: dynamic weights based on actual server load

2009-10-17 Thread Angelo Höngens
Just read this thread, and I thought I would give my humble opinion on this:

As a hosting provider we use both windows and unix backends, en we use haproxy 
to balance requests across sites on a per-site backend (with squid in front of 
haproxy). What I would love to see, is dynamic balancing based on the 
round-trip time of the health check.

So when a backend is slower to respond, the weight should go down (slowly), so 
the faster servers would get more requests. Now that's a feature I'd love to 
see.. And then there would not be anything to configure on the backend (we 
don't always have control over the backend application)

Just my 2 cents.
-- 

 
With kind regards,
 
 
Angelo Höngens
 
Systems Administrator
 
--
NetMatch
tourism internet software solutions
 
Ringbaan Oost 2b
5013 CA Tilburg
T: +31 (0)13 5811088
F: +31 (0)13 5821239
 
mailto:a.hong...@netmatch.nl
http://www.netmatch.nl
--

-Original Message-
From: Willy Tarreau [mailto:w...@1wt.eu] 
Sent: vrijdag 16 oktober 2009 22:59
To: Hank A. Paulson
Cc: Craig; haproxy@formilux.org
Subject: Re: dynamic weights based on actual server load

On Fri, Oct 16, 2009 at 01:36:52PM -0700, Hank A. Paulson wrote:
 For the code you are developing, if you make the interface general enough 
 so that parameters can be added or removed that would be good. 
 Telnet/text/memcached style protocols seem popular to allow easy 
 debugging/monitoring.
 So if your protocol says a machine has to send a load info bundle like:
 SS:8cbed340118ddf87e2d8ca4352006572
 SYSID: blah1
 SAMPLETIME: 2009-10-14-22-00-03
 CPU: 83.23343455
 NETI: 134238.0232
 NETO: 492283.6549
 DISK: 433.232
 ES:8cbed340118ddf87e2d8ca4352006572

It's a lot better to return these information in HTTP headers, because
that can be added on top of any other resource that haproxy would check.

(...)
 It is probably possible to just modify feedbackd to emit haproxy set 
 weight commands.

Don't do that ! The set weight is for the human, not for an automaton.
The most interesting usage is set weight xx/xx 0 to disable a server
before operating on it, or set weight xx/xx 50% to off-load it a bit
if you see it start swapping. If you send an automated tool on it, it
will constantly change your manually assigned values.

 More interesting, I think would be to combine a multiple load parameter 
 (active connections, CPU, net in/out bytes, net in/out packets, disk io, 
 etc) feedback system with the ideas from the NetBSD neural network 
 scheduler, creating an ai based dynamic load balancing system.
 http://softlayer.dl.sourceforge.net/project/nnsched/docs/thesis/nnsched.pdf
 This is more possible now that we have multi core systems that would have 
 some idle CPU resources available for the ai compute load.

if doing some computations on that few parameters consumes even
a measurable amount of CPU, then they are clearly wrong. We're
not doing fractals here. Combining 3-4 parameters should not
results on billions of operations.

Willy





Re: round robin

2009-08-18 Thread Angelo Höngens

On 18-8-2009 14:08, Johan Duflost wrote:

Hello,
 
We have a farm of asp.net web servers behind haproxy configured with 
round-robin.

It seems to work but in fact the requests are not evenly distributed.
All the servers have the same capacity and there's no priority declared 
in the haproxy configuration file.
Does anybody know why it doesn't seem possible to have a real 
distribution accross the servers?



Well, it's very well possible, but you might have a configuration that 
tells haproxy to do something else..


Are you sending httpclose responses? Are you using session cookies?


--


With kind regards,


Angelo Höngens
systems administrator

MCSE on Windows 2003
MCSE on Windows 2000
MS Small Business Specialist
--
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
+31 (0)13 5811088
+31 (0)13 5821239

a.hong...@netmatch.nl
www.netmatch.nl
--





Re: round robin

2009-08-18 Thread Angelo Höngens

On 18-8-2009 16:34, Johan Duflost wrote:

Hello,

Here is an excerpt of our config file :

appsession ASP.NET_SessionId len 52 timeout 1h
cookie ASP.NET_SessionId prefix
option persist
balance roundrobin
option forwardfor
option httpclose
server 1 xx.xx.xx.xx:80 cookie 1 check inter 3000 rise 2 fall 3
server 2 xx.xx.xx.xx:80 cookie 2 check inter 3000 rise 2 fall 3
server 3 xx.xx.xx.xx:80 cookie 3 check inter 3000 rise 2 fall 3

Did you see anything special that could explain this behavior?


Well, to my best knowledge (but please correct me if I'm wrong), the 
'cookie' option make sure you have session stickyness.


With session stickyness, you don't have perfect balancing, since, well.. 
users stick around :-) Some user's sessions take 10 seconds, others will 
take 10 minutes, and only new users (which) don't have a cookie will be 
round-robin sent to a new server.


If you do want perfect balancing, you have to turn off stickyness. But 
that might mean you have to adapt your ASP.NET application to non-sticky 
sessions (if you need server-side user sessions, you'll have to do 
something like storing the sessions in a central db, and identifying 
users by cookies or something like that, like all big players do).



--


With kind regards,


Angelo Höngens
systems administrator

MCSE on Windows 2003
MCSE on Windows 2000
MS Small Business Specialist
--
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
+31 (0)13 5811088
+31 (0)13 5821239

a.hong...@netmatch.nl
www.netmatch.nl
--





RE: HAProxy not writing to logs.

2009-07-20 Thread Angelo Höngens
I'm running almost the same config here (FreeBSD 7.2 instead of 7.0), and it's 
working fine here:


--from /etc/rc.conf:
apache22_enable=YES
syslogd_flags=-a 127.0.0.1:*

--from /etc/syslog.conf
local0.* /var/log/haproxy.log


$ ps auxw | grep syslog
root 1505  0.0  0.0  5692  1204  ??  Ss   Fri05PM   0:01.72 
/usr/sbin/syslogd -a 127.0.0.1:*

in my haproxy.conf is:
global
  log 127.0.0.1   local0 notice

-- 


With kind regards,


Angelo Höngens

Systems Administrator

--
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
T: +31 (0)13 5811088
F: +31 (0)13 5821239

mailto:a.hong...@netmatch.nl
http://www.netmatch.nl
--

From: William Angus [mailto:w...@virtualxpert.com.au] 
Sent: maandag 20 juli 2009 9:21
To: haproxy@formilux.org
Subject: HAProxy not writing to logs.

Good afternoon,

I am unable to configure HAProxy to write to local logs on the server the 
process is running on..
I am currently configuring two load-balancers in failover using Common Address 
Redundancy Protocol (CARP) to service a pair of application servers through 
ports 8080 and 8443. Aside from writing to log files the system appears to be 
running well with both http and https traffic passing through to the backend 
servers.
Both proxy servers are configured identically with the following 
details/settings. The intention is to have a utility parse the logs and send 
status and/or warning emails as needed.

Server OS
FreeBSD 7.0-release amd64 with CARP compiled into the kernel.
--CARP is used for availability. The interface monitors the ip that is used to 
connect to the app servers.

HAProxy version
1.3.18 from ports/net/haproxy-devel/

--From /etc/syslog.conf
+127.0.0.1
local0.*    /var/log/haproxy.log
--I have tried this with +127.0.0.1, +localhost, both and neither. 
/var/log/haproxy.log is owned by haproxy user (tested when running process as 
root but still didn't write to the log when owned by root or haproxy user, root 
wouldn't be used live of course)

--From /etc/rc.conf
syslogd_enable=YES
syslogd_flags=-a 127.0.0.1/24:* -4 -b 127.0.0.1
haproxy_enable=YES
--Have tried the -a and -b entries seperately and -b localhost. Also with and 
without -4 (I doubt this is strictly necessary as ip6 isn't currently 
configured on any interfaces so there would be no conflict).

 ps -auxw | grep syslog
root   627  0.0  0.1  4684  1380  ??  Ss    1:24PM   0:00.02 
/usr/sbin/syslogd -a 127.0.0.1/24:* -4 -b 127.0.0.1
--So the syslogd is running with the appropriate syslogd_flags (-b could also 
be used as localhost?) PID is different on second server but that is to be 
expected.

--/usr/local/etc/haproxy.conf - only changes from running are any ips, 
server-names or user/pass. This is a simpler one than I will be using in 
production. Intention is that the process will be jailed for security and other 
bits will be tweaked as needed. This is just a start-testing config.
global
    maxconn 4096
    pidfile /var/run/haproxy.pid
    daemon
    user haproxy
    group haproxy
    log 127.0.0.1 local0 info

defaults
    mode http
    retries 3
    option redispatch
    stats enable
    stats auth user:password
    maxconn 2000
    contimeout 5000
    clitimeout 5
    srvtimeout 5

listen name IP:8080 - this is the ip used by CARP interface
    log global
    mode http
    option httplog
    cookie name insert
    balance roundrobin
    option httpclose
    option forwardfor
    server server1 IP:8080 cookie name check
    server server2 IP:8080 cookie name check

listen name-secure IP:8443 - this is the ip used by CARP interface
    log global
    option tcplog
    mode tcp
    balance roundrobin
    option httpclose
    option forwardfor
    server server1 IP:8443 check
    server server2 IP:8443 check

The ip-address that the CARP interfaces monitor has an appropriate DNS record.
I am aware that the FreeBSD version of syslog doesn't support the -r tag that 
some Linux versions do and was wondering, if I can't get the process 
communicating over a socket is there an equivalent that I could use. 

Any ideas as to what I am missing?

Kind regards
William Angus

-- 


Consulting . Enterprise Architecture . Custom Software
Will Angus - Systems Analyst
Virtual Xpert Pty. Ltd.
Suite 207, 35 Buckingham St.
Surry Hills, NSW  2010
Australia
Ph: 
+61 2 9007 1480
Fax: 
+61 2 8905 9438
Mob: 
+61 424 298 449
Email: 
w...@virtualxpert.com.au
WWW: 
www.virtualxpert.com.au
ABN: 
21 118 797 354






Re: stats are cut off

2009-06-23 Thread Angelo Höngens

On 22-6-2009 16:22, Krzysztof Oledzki wrote:
Which version? Does it always break at the same position? Could you share 
your config with us? ;)


I am running FreeBSD 7.2, and was running haproxy 1.2.18, from the ports 
(net/haproxy).


Now I installed haproxy 1.3.18 from the ports (net/haproxy-devel), and 
the problem no longer occurs.


By the way, nice redesign of the stats page, it looks much better! :)


I always like to use the FreeBSD ports versions, then I use a single 
framework to manage all my software, and I am automatically reminded to 
upgrade when there's a new version of the port.


Any idea when the 'net/haproxy' port wil be upgraded to 1.3.x?



--


With kind regards,


Angelo Höngens
systems administrator

MCSE on Windows 2003
MCSE on Windows 2000
MS Small Business Specialist
--
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
+31 (0)13 5811088
+31 (0)13 5821239

a.hong...@netmatch.nl
www.netmatch.nl
--





stats are cut off

2009-06-22 Thread Angelo Höngens
Hey guys and girls,

I'm a happy user of HAProxy, and for one of my new projects I'm running into a 
small problem.

I have a new configuration with a 120 different instances (one instance for 
every site on a couple of servers), and it looks like everything works fine. 
However, I have a problem with the stats page, it will only show the first 38,5 
sites :)

Please see this screenshot: http://files.hongens.nl/2009/06/22/haproxystats.png

Somewhere it's cut off after a /tr tag. This is kind of annoying, because I 
use the stats page to see the status of the web server nodes behind HAProxy.

Don't see anything interesting in the logs..

Any ideas anyone?

-- 


With kind regards,


Angelo Höngens

Systems Administrator

--
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
T: +31 (0)13 5811088
F: +31 (0)13 5821239

mailto:a.hong...@netmatch.nl
http://www.netmatch.nl
--





stats are cut off

2009-06-22 Thread Angelo Höngens
Hey guys and girls,

I'm a happy user of HAProxy, and for one of my new projects I'm running into a 
small problem.

I have a new configuration with a 120 different instances (one instance for 
every site on a couple of servers), and it looks like everything works fine. 
However, I have a problem with the stats page, it will only show the first 38,5 
sites :)

Please see this screenshot: http://files.hongens.nl/2009/06/22/haproxystats.png

Somewhere it's cut off after a /tr tag. This is kind of annoying, because I 
use the stats page to see the status of the web server nodes behind HAProxy.

Don't see anything interesting in the logs..

Any ideas anyone?


-- 


With kind regards,


Angelo Höngens

Systems Administrator

--
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
T: +31 (0)13 5811088
F: +31 (0)13 5821239

mailto:a.hong...@netmatch.nl
http://www.netmatch.nl
--