stick tables and url_param + post headers - counter‏
Hello, I am trying to rate limit requests depending on their specific identifier which is sent either as a post header or a query string parameter. Below is my starting config (am i mistaken to be using this ? ) stick-table type string len 70 size 5M expire 1m store gpc0_rate(60s),conn_cnt,conn_cur,conn_rate(60s),sess_cnt,sess_rate(60s),http_req_rate(60s) stick on url_param(uid) my hope is to use a throttled backend, if connections within 1 minute from the same UID (query string or post header) exceeds 30 I have the same setup working with IPs though im finding it a bit tricky to do the same with qs/headers Any advice on the right direction? i am not confident with the above counters
RE: rate limiting according to "total time" - possible ?
That's exactly what i wanted!! thank you willy > Date: Mon, 14 Sep 2015 07:38:08 +0200 > From: w...@1wt.eu > To: r_o_l_a_...@hotmail.com > CC: haproxy@formilux.org > Subject: Re: rate limiting according to "total time" - possible ? > > Hi Roland, > > On Fri, Sep 11, 2015 at 05:11:11PM +0300, Roland RoLaNd wrote: > > hello > > i have haproxy directing traffic to a number of backends. > > these backends can auto scale upon traffic; my goal is to change "maxcon" > > depending on "total time" or "backend time" that a request took to > > complete. > > for example: > > if totaltime < 1 second ; maxcon = 1000if totaltime < 2 second: maxconn = > > 500etc... > > > > the goal is to hold connections in queue till backend auto scaling is in > > effect. > > > > Can i do the above scenario within haproxy config or a cron that checks > > haproxy socket/totaltime and act accordingly is a better idea? > > > > do you have an alternative advice for me to accomplish that goal ? > > I could be wrong, but I think you're trying to re-implement by hand the > dynamic rate limiting you can get using minconn,maxconn and fullconn. It > dynamically increases or decreases the effective per-server maxconn > depending on the total number of connections on all servers in the > backend so that queues decrease when connection count increases. > > Willy > >
rate limiting according to "total time" - possible ?
hello i have haproxy directing traffic to a number of backends. these backends can auto scale upon traffic; my goal is to change "maxcon" depending on "total time" or "backend time" that a request took to complete. for example: if totaltime < 1 second ; maxcon = 1000if totaltime < 2 second: maxconn = 500etc... the goal is to hold connections in queue till backend auto scaling is in effect. Can i do the above scenario within haproxy config or a cron that checks haproxy socket/totaltime and act accordingly is a better idea? do you have an alternative advice for me to accomplish that goal ? Thanks in advance
Change route on http_err_cnt
Stick table /request tracking inquiry: is it possible to traffic to a different backend when http error (5xx/4xx) goes higher than a certain threshold ? my config is as such: acl phoenix_bound path_beg -i -f /etc/haproxy/phoenix_bound.lst use_backend phoenix if phoenix_bound My end goal is to accomplish this condition IF (http status code) from (phoenix backend) 399 count 100 ; then use_backend catch_all for al subsequent requests till there are no more errors I am reading up on stick tables and tracking ; but i am confused with the enormous things that can be done with such options so a nudge in the right direction would be greatly appreciated Another question if possible: is it possible to set counters for query string parameters ? for example counting how many requests a specific user id have made to a specific backend
stick-table and conn_rate question
managed to successfully reject access from specific users depending on
condition; but what i eventually want is to provide them with a certain page
instead of reject (redirect isn't an option)
backend phoenix
stick-table type string len 40 size 5M expire 2m store
conn_rate(60s)tcp-request inspect-delay 10sstick on url_param(sid) table
phoenixtcp-request content track-sc0 url_param(sid)errorfile 200
/etc/haproxy/custom_response/phoenix.http if { sc0_conn_rate gt 10 }
checking socket; the conn rate is above 10:0x8581a0:
key=100testing01 use=0 exp=119272 server_id=1 conn_rate(6)=90
i think the problem is that condition should be set in frontend config in a
way that points to the phoenix table instead of the default frontend table...
any advice?
frequent NOSRV/SC log hits behind AWS ELB
Hello,
i am running haproxy version: 1.5.11 on EC2 instances behind an AWS load
balancer
lately i am noticing a lot of 503 forbidden logs with SC as termination state
due to nosrv error
my backend servers(which are behind an ELB of their own) are all healthy and
responsive
moreover i set a loop that checks port 80 between haproxy and backend servers;
and it never failed; it was checking the connection every 10 ms
this is a log sample:
Mar 10 10:33:50 api haproxy[1056]: 172.16.100.169:15235
[10/Mar/2015:10:33:50.905] API API/NOSRV 8/-1/-1/-1/8 503 213 - - SC--
79/79/0/0/0 0/0 {177.103.215.19|Dalvik/1.6.0 (Linux; U; Android 4.4.4; XT1032
Build/KXB21.14-L1.} POST /api/v2.3/androidevent?buildnumber=1.10 HTTP/1.1
and this is my current config:
globallog /dev/loglocal0log /dev/loglocal1 notice
chroot /var/lib/haproxystats socket /run/haproxy/admin.sock mode 660
level adminstats timeout 30suser haproxygroup haproxy
maxconn 65000daemon
# Default SSL material locationsca-base /etc/ssl/certs
crt-base /etc/ssl/private
# Default ciphers to use on SSL-enabled listening sockets.# For
more information, see ciphers(1SSL).ssl-default-bind-ciphers
kEECDH+aRSA+AES:kRSA+AES:+AES256:RC4-SHA:!kEDH:!LOW:!EXP:!MD5:!aNULL:!eNULL
ssl-default-bind-options no-sslv3
defaultslog globalmodehttpoption httplog
option dontlognulltimeout connect 1timeout client 5
timeout server 5# users which we are redrecting no where,
example rejected will die in 50 mstimeout tarpit 50errorfile
400 /etc/haproxy/errors/400.httperrorfile 403
/etc/haproxy/errors/403.httperrorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.httperrorfile 502
/etc/haproxy/errors/502.httperrorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.httpbalance roundrobin
# keeps keep alive between client and proxy but disable it between proxy and
backednoption http-server-closeoption forwardfor
option redispatch retries 99 frontend API bind *:80
maxconn 6 # Blacklist: Deny access to some IPs before
anything else is checkedtcp-request content reject if { src -f
/etc/haproxy/blacklist.lst } http-request set-header
X-custom-http-scheme %[hdr(X-Forwarded-Proto)]
stick-table type ip size 500k expire 30s store
conn_cur,conn_rate(10s),http_req_rate(10s),http_err_rate(10s)
option http-server-close# elb logs pubc ips capture
request header X-Forwarded-For len 50capture request header User-Agent
len 64acl network_allowed src x.x.x.xacl
restricted_page path_beg /restrictedhttp-request deny if
restricted_page !network_allowed# direct uris to
propper elbacl uri_api path_beg /apiacl
uri_wdev path_beg /wdevacl uri_staging path_beg /staging
use_backend api if uri_apiuse_backend wdev if
uri_wdevuse_backend staging if uri_staging
default_backend API
backend APIserver API ELB_CNAME:80 check backend wdevserver
wdev ELB_CNAME:80 check backend stagingserver staging ELB_CNAME:80
check
