Haproxy core utilization
Hello, i noticed that sometimes (if i monitor the server using top for a few minutes) one haproxy process seem to have 99% utilization while the other three are about 25% How can i avoid this behavior ? (how can i force haproxy not to use core 0) Ubuntu 12.04.5 LTS Haproxy 1.5.9 nproc 4 -- Haim Ari / SysOps Manager M: 972.584563032 / T: 972.722288367 [cid:part1.02070805.07060601@startapp.com]
HAProxy for Statis IP redundancy
Hi - Would you be able to tell me if HAProxy can be used to solve the following problem? I host an iis 7.5) windows site on a comcast business static IP (in office). the internet goes down sometimes and I'd like redundancy. I cant find the proper way to add a second internet provider/static IP for failover when the primary line goes down. I thought maybe DNS round robin but it looks like an IIS site cannot have multiple bindings for this Any thoughts? Is there a standard architecture or method for Internet connectivity redundancy for one website on one server ? Cost is an issue. Thanks! Mitchell Visualjobmatch.com
[PATCH] DOC: match several lua configuration option names to those implemented in code
Hi, Ive found some inconsistencies in the documentation, patch attached. Could you take a look and merge it? Thanks. Regards, PiBa-NL From 007f377f637dbafc47cb77f6650e4df55e08b608 Mon Sep 17 00:00:00 2001 From: Pieter Baauw piba.nl@gmail.com Date: Sun, 16 Aug 2015 15:26:24 +0200 Subject: [PATCH] DOC: match several lua configuration option names to those implemented in code --- doc/configuration.txt | 2 +- doc/lua-api/index.rst | 18 +- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/doc/configuration.txt b/doc/configuration.txt index 424b31d..83f337d 100644 --- a/doc/configuration.txt +++ b/doc/configuration.txt @@ -1456,7 +1456,7 @@ It is possible to send email alerts when the state of servers changes. If configured email alerts are sent to each mailer that is configured in a mailers section. Email is sent to mailers using SMTP. -mailer mailersect +mailers mailersect Creates a new mailer list with the name mailersect. It is an independent section which is referenced by one or more proxies. diff --git a/doc/lua-api/index.rst b/doc/lua-api/index.rst index 26641ba..8671f3e 100644 --- a/doc/lua-api/index.rst +++ b/doc/lua-api/index.rst @@ -514,7 +514,7 @@ Channel class If the buffer cant receive more data, a 'nil' value is returned. :param class_channel channel: The manipulated Channel. - :returns: a string containig the avalaiable line or nil. + :returns: a string containing the available line or nil. .. js:function:: Channel.set(channel, string) @@ -579,21 +579,21 @@ HTTP class This class contain all the HTTP manipulation functions. -.. js:function:: HTTP.req_get_header(http) +.. js:function:: HTTP.req_get_headers(http) Returns an array containing all the request headers. :param class_http http: The related http object. :returns: array of headers. - :see: HTTP.res_get_header() + :see: HTTP.res_get_headers() -.. js:function:: HTTP.res_get_header(http) +.. js:function:: HTTP.res_get_headers(http) Returns an array containing all the response headers. :param class_http http: The related http object. :returns: array of headers. - :see: HTTP.req_get_header() + :see: HTTP.req_get_headers() .. js:function:: HTTP.req_add_header(http, name, value) @@ -661,9 +661,9 @@ HTTP class :param class_http http: The related http object. :param string name: The header name. :param string value: The header value. - :see: HTTP.req_set_header() + :see: HTTP.req_rep_header() -.. js:function:: HTTP.req_replace_header(http, name, regex, replace) +.. js:function:: HTTP.req_rep_header(http, name, regex, replace) Matches the regular expression in all occurrences of header field name according to regex, and replaces them with the replace argument. The @@ -674,9 +674,9 @@ HTTP class :param string name: The header name. :param string regex: The match regular expression. :param string replace: The replacement value. - :see: HTTP.res_replace_header() + :see: HTTP.res_rep_header() -.. js:function:: HTTP.res_replace_header(http, name, regex, string) +.. js:function:: HTTP.res_rep_header(http, name, regex, string) Matches the regular expression in all occurrences of header field name according to regex, and replaces them with the replace argument. The -- 1.9.5.msysgit.1
[PATCH] BUILD: add USE_LUA to BUILD_OPTIONS when it's used
haproxy -vv doesn't indicate that USE_LUA was specified at compilation time.
This is caused by the Makefile, which doesn't update BUILD_OPTIONS.
---
Makefile | 1 +
1 file changed, 1 insertion(+)
diff --git a/Makefile b/Makefile
index 6ab7d54..63ac6ef 100644
--- a/Makefile
+++ b/Makefile
@@ -595,6 +595,7 @@ endif
ifneq ($(USE_LUA),)
check_lua_lib = $(shell echo int main(){} | $(CC) -o /dev/null -x c - $(2)
-l$(1) 2/dev/null echo $(1))
+BUILD_OPTIONS += $(call ignore_implicit,USE_LUA)
OPTIONS_CFLAGS += -DUSE_LUA $(if $(LUA_INC),-I$(LUA_INC))
LUA_LD_FLAGS := $(if $(LUA_LIB),-L$(LUA_LIB))
ifeq ($(LUA_LIB_NAME),)
--
2.5.0
[PATCH] MINOR cfgparse: Correct the mailer warning text to show the right names to the user
Hi Guys,
Patch attached to correct the mailer warning text to show the right
names to the user.
Regards,
PiBa-NL
From aa2cccdf5e95d2850692ec8189fc9ed20a586575 Mon Sep 17 00:00:00 2001
From: Pieter Baauw piba.nl@gmail.com
Date: Mon, 17 Aug 2015 00:45:05 +0200
Subject: [PATCH] MINOR cfgparse: Correct the mailer warning text to show the
right names to the user
---
src/cfgparse.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/src/cfgparse.c b/src/cfgparse.c
index 98ccd5d..34d029b 100644
--- a/src/cfgparse.c
+++ b/src/cfgparse.c
@@ -7319,9 +7319,9 @@ int check_config_validity()
if (curproxy-email_alert.set) {
if (!(curproxy-email_alert.mailers.name
curproxy-email_alert.from curproxy-email_alert.to)) {
Warning(config : 'email-alert' will be ignored for
%s '%s' (the presence any of
- 'email-alert from', 'email-alert level'
'email-alert mailer',
- 'email-alert hostname', or 'email-alert
to'
- requrires each of 'email-alert from',
'email-alert mailer' and 'email-alert'
+ 'email-alert from', 'email-alert level'
'email-alert mailers',
+ 'email-alert myhostname', or 'email-alert
to'
+ requires each of 'email-alert from',
'email-alert mailers' and 'email-alert to'
to be present).\n,
proxy_type_str(curproxy), curproxy-id);
err_code |= ERR_WARN;
--
1.9.5.msysgit.1
Re: HAProxy for Statis IP redundancy
On 16/08/2015 11:21 PM, Mitchell Gurspan mitch...@visualjobmatch.com wrote: Hi – Would you be able to tell me if HAProxy can be used to solve the following problem? I host an iis 7.5) windows site on a comcast business static IP (in office). the internet goes down sometimes and I’d like redundancy. I cant find the proper way to add a second internet provider/static IP for failover when the primary line goes down. I thought maybe DNS round robin but it looks like an IIS site cannot have multiple bindings for this Any thoughts? Is there a standard architecture or method for Internet connectivity redundancy for one website on one server ? Cost is an issue. Thanks! Mitchell Visualjobmatch.com Can't see what this has to do with haproxy this is something you setup in your infrastructure. Get a router with two WAN ports each connected to different ISP. For DYI you can set linux box as router with iptables and policy routing. Google will show you many exmples how to do it.
Re: IP address ACLs
Sounds good. If I use the external file, will HAProxy reload it if the modification timestamp changes? Or do I need to explicitly send a reload signal? On Sat, Aug 15, 2015 at 3:39 AM, Baptiste bed...@gmail.com wrote: Hi, there is no performance drop of loading from a file or directly in the config file. That said, if you have multiple ACLs with the same name loading many IPs, then you'll perform as many lookups as you have ACLs... While loading content from a file would perform a single lookup. Anyway, there should not be any noticeable performance impact, since IP lookup is very quick in HAProxy (a few hundred of nano second in a tree of 1.000.000 IPs). Concerning comments, any string after a dash '#' is considered as a comment and not loaded in the ACL. Baptiste On Sat, Aug 15, 2015 at 8:28 AM, Nathan Williams nath.e.w...@gmail.com wrote: We use a file for about 40 cidr blocks, and don't have any problems with load speed. Presumably large means more than that, though. We use comments as well, but they have to be at the beginning of their own line, not tagged on after the address. On Fri, Aug 14, 2015, 9:09 PM CJ Ess zxcvbn4...@gmail.com wrote: When doing a large number of IP based ACLs in HAProxy, is it more efficient to load the ACLs from a file with the -f argument? Or is just as good to use multiple ACL statements in the cfg file? If I did use a file with the -f parameter, is it possible to put comments in the file?
Re: Regarding using HAproxy for rate limiting
Hi Baptiste,I tried to read about SC0 and SRC, but i am not quite sure what i
would gain by changing SRC to SCO for the acl paramters? did u have some
example to explain?
Thanks
From: Amol mandm_z...@yahoo.com
To: Baptiste bed...@gmail.com
Cc: HAproxy Mailing Lists haproxy@formilux.org
Sent: Friday, August 14, 2015 2:06 PM
Subject: Re: Regarding using HAproxy for rate limiting
Hi Baptiste,Yes sorry i might have confused you with some questions but to
answer your questions
here, the question is: what kiils your server exactly?
A high number of queries from a single users or whatever the number of users?
I'm trying to understand what you need...Yes i am trying to protect against
high number of requests from a single user who can use API's or even
mis-configure API's to generate high load.
reposting the configuration
frontend www-https
bind xx.xx.xx.xx:443 ssl crt .pem ciphers AES128+EECDH:AES128+EDH
no-sslv3 no-tls-tickets
# Table definition
stick-table type ip size 100k expire 30s store
gpc0,conn_cur,conn_rate(3s),http_req_rate(10s),http_err_rate(10s)
# Allow clean known IPs to bypass the filter
tcp-request connection accept if { src -f /etc/haproxy/whitelist.lst }
# this is sending data defined in the stick-table and storing it the
stick-table since by default nothing is restored in it
tcp-request connection track-sc0 src
# Shut the new connection as long as the client has already 40 opened
tcp-request connection reject if { src_conn_cur ge 40 }
# if someone has more than 40 connections in over a period of 3 seconds,
REJECT
tcp-request connection reject if { src_conn_rate ge 40 }
# tracking connections that are not rejected from clients that don't have 10
connections/don't have 10 connections/3 seconds
#tcp-request connection reject if { src_get_gpc0 gt 0 }
acl abuse_err src_http_err_rate ge 10
acl flag_abuser_err src_inc_gpc0 ge 0
acl abuse src_http_req_rate ge 250
#acl flag_abuser src_inc_gpc0 ge 0
#tcp-request content reject if abuse_err flag_abuser_err
#tcp-request content reject if abuse flag_abuser
use_backend backend_slow_down if abuse flag_abuser
use_backend backend_slow_down if abuse_err flag_abuser_err
default_backend www-backend
backend www-backend
balance leastconn
cookie BALANCEID insert indirect nocache secure httponly
option httpchk HEAD /xxx.php HTTP/1.0
redirect scheme https if !{ ssl_fc }
server A1 xx.xx.xx.xx:80 cookie A check
server A2 yy.yy.yy.yy:80 cookie B check
backend backend_slow_down
timeout tarpit 2s
errorfile 500 /etc/haproxy/errors/429.http
http-request tarpit
--
Yes i will check out the difference between SC0 and SRC paramters in config
regarding this .
What i am doing here is that if the http_req_rate 250 then i want to send
them to a another backend which gives them a rate limiting message or if the
number of concurrent connections are 4, then i want to rate limit their
usage and allow on 40 connections to come in.
i was trying to make 2 points i guess i should have been more clear...So i was
saying that based on my config i am trying to achieve 2 things
1) to rate limit a client with high number of http requests in a certain time
span (http_req_rate)2) to rate limit a client with high number of concurrent
connections in the certain time span. (src_conn_cur and src_conn_rate )
Thanks once again for looking into this.
From: Baptiste bed...@gmail.com
To: Amol mandm_z...@yahoo.com
Cc: HAproxy Mailing Lists haproxy@formilux.org
Sent: Friday, August 14, 2015 1:40 PM
Subject: Re: Regarding using HAproxy for rate limiting
Hi Amol,
On Fri, Aug 14, 2015 at 4:16 PM, Amol mandm_z...@yahoo.com wrote:
Hello,
I am been trying to configure my Haproxy for rate limiting our customer
usage, and wanted to know/understand some of my options
what i am trying to achieve is to throttle any clients requests/api calls
that can take lead to high load and can kill my servers.
here, the question is: what kiils your server exactly?
A high number of queries from a single users or whatever the number of users?
I'm trying to understand what you need...
First of all here is my configuration i have so far from reading a few
articles
frontend www-https
bind xx.xx.xx.xx:443 ssl crt .pem ciphers AES128+EECDH:AES128+EDH
no-sslv3 no-tls-tickets
# Table definition
stick-table type ip size 100k expire 30s store
gpc0,conn_cur,conn_rate(3s),http_req_rate(10s),http_err_rate(10s)
# Allow clean known IPs to bypass the filter
tcp-request connection accept if { src -f /etc/haproxy/whitelist.lst }
# this is sending data defined in the stick-table and storing it the
stick-table since by default nothing is restored in it
tcp-request connection track-sc0 src
# Shut the new connection as long as the client has already 10 opened
tcp-request
Re: Forwarding issue
Thanks for your help. So, it appears that in certain instances a POST is
done using relative path or URN. The question is how do I tell haproxy
that this is related to the original url that is found on the certificate.
Meaning, this part I need to change so that the forwarding is done based on
CN in certificate. Or, perhaps, there is another way to do it.
use_backend bk_remail_cert if { ssl_fc_sni remail.mydomain.com } # content
switching based on SNI
On Thu, Aug 13, 2015 at 12:21 AM Jeff Palmer j...@palmerit.net wrote:
The NOSRV in the logs you posted, means that haproxy doesn't believe
there is a server available on the backend to fulfull the request.
The first request hits the bk_remail_cert backend, but the next two
requests don't, my guess is that the requests don't match your
use_backend requirements, and you don't have a default_backend set
for the request to fall through. thus, the NOSRV logs.
On Wed, Aug 12, 2015 at 10:58 PM, Roman Gelfand rgelfa...@gmail.com
wrote:
I am not seeing a response to the following post. The apache backend
server
show now trace of the post request, below.
Aug 12 22:43:43 localhost haproxy[15606]: 38.105.236.253:59539
[12/Aug/2015:22:43:12.935] farm_test_ssl~ bk_remail_cert/remail
26582/0/1/3530/30113 200 1229 - - 1/1/0/0/0 0/0 POST
/services/ajax.php/imp/poll HTTP/1.1
Aug 12 22:43:44 localhost haproxy[15606]: 107.107.59.175:29935
[12/Aug/2015:22:43:44.289] farm_test_ssl~ farm_test_ssl/NOSRV
-1/-1/-1/-1/86 503 212 - - SC-- 1/1/0/0/0 0/0 POST
/Microsoft-Server-ActiveSync?Cmd=SyncUser=ruser%40mydomain.com
DeviceId=SEC15CCC18A02315DeviceType=SAMSUNGSMG900A
HTTP/1.1
Aug 12 22:43:57 localhost haproxy[15606]: 107.107.59.175:17798
[12/Aug/2015:22:43:57.550] farm_test_ssl~ farm_test_ssl/NOSRV
-1/-1/-1/-1/88 503 212 - - SC-- 1/1/0/0/0 0/0 POST
/Microsoft-Server-ActiveSync?Cmd=SyncUser=ruser%40mydomain.com
DeviceId=SEC15CCC18A02315DeviceType=SAMSUNGSMG900A
HTTP/1.1
this is the front-end back-end configuration
frontend farm_test_ssl
mode http
bind 0.0.0.0:443 ssl crt crt /etc/ssl/certs/remail.pem
use_backend bk_remail_cert if { ssl_fc_sni remail.mydomain.com } #
content
switching based on SNI
backend bk_remail_cert
mode http
server remail 192.168.8.166:80 check
http-request set-header X-Forwarded-Proto https if { ssl_fc }
On Wed, Aug 12, 2015 at 5:12 PM Jeff Palmer j...@palmerit.net wrote:
Describing the issues you think it's causing fire haproxy would also be
helpful.
On Aug 12, 2015 4:37 PM, Baptiste bed...@gmail.com wrote:
On Wed, Aug 12, 2015 at 6:34 PM, Roman Gelfand rgelfa...@gmail.com
wrote:
Why would the following apache directives cause problems for haproxy.
RewriteRule ^/Microsoft-Server-ActiveSync /rpc.php
[PT,L,QSA]
RewriteRule .* -
[E=HTTP_MS_ASPROTOCOLVERSION:%{HTTP:Ms-Asprotocolversion}]
RewriteRule .* -
[E=HTTP_X_MS_POLICYKEY:%{HTTP:X-Ms-Policykey}]
RewriteRule .* -
[E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
Thanks in advance
First, you say 'hi'
Second, you explain your problem and whayt those apache rules are
supposed to do, what type of application are they applied to and how
this application is supposed to work.
Without a bit of context, it is impossible to help!
Baptiste
--
Jeff Palmer
https://PalmerIT.net
Re: [PATCH] BUILD: add USE_LUA to BUILD_OPTIONS when it's used
On Sun, Aug 16, 2015 at 08:27:38PM +0200, Cyril Bonté wrote: haproxy -vv doesn't indicate that USE_LUA was specified at compilation time. This is caused by the Makefile, which doesn't update BUILD_OPTIONS. Patch applied, thank you Cyril. Willy
