Hi Dennis, Answering inline in your email.
> Question 1: Is there a better way to reset the gpc0 counter other than > waiting for the stick-table entry to expire? > > In my test if I hit haproxy with the load-testing tool apache bench to > trigger the 10 req/s limit for two seconds and then follow that up with > a pattern of 1 req/s for a minute these requests will never succeed > because gpc0 is greater than zero, will never reset and the stick-table > entry will never expire because the timer will always get reset by the 1 > req/s pattern so the user is effectively locked out forever even though > he is no longer exceeding the request/s limit. > > Wouldn't it be better to reset the gpc0 counter to zero once > http_req_rate has dropped below 10 again to not create this kind of > perma-block? yes, you can, there is a sample called sc0_clr_gpc0 whose purpose is to clear the value of gpc0. an other solution would not to measure gpc0 itself but its growing rate using sc0_gpc0_rate. growing would be very low with 1 request per minute. > Question 2: When I use wrk instead of ab it seems the request limiting > doesn't work at all. What wrk does is it doesn't create new connections > for each request but only creates a bunch of connections initially and > then sends all requests using these permanent connections. These are a > couple of stick-table dumps I did after starting the wrk test: > > 0xe5e854: key=10.99.0.1 use=10 exp=7791 gpc0=15771 conn_cur=10 > http_req_rate(10000)=15780 > 0xe5e854: key=10.99.0.1 use=10 exp=7247 gpc0=19767 conn_cur=10 > http_req_rate(10000)=19776 > 0xe5e854: key=10.99.0.1 use=10 exp=6727 gpc0=23606 conn_cur=10 > http_req_rate(10000)=23615 > 0xe5e854: key=10.99.0.1 use=10 exp=6247 gpc0=26718 conn_cur=10 > http_req_rate(10000)=26727 > 0xe5e854: key=10.99.0.1 use=10 exp=5823 gpc0=29760 conn_cur=10 > http_req_rate(10000)=29769 > 0xe5e854: key=10.99.0.1 use=10 exp=5424 gpc0=32622 conn_cur=10 > http_req_rate(10000)=32631 > 0xe5e854: key=10.99.0.1 use=10 exp=4967 gpc0=35964 conn_cur=10 > http_req_rate(10000)=35973 > 0xe5e854: key=10.99.0.1 use=10 exp=4567 gpc0=38779 conn_cur=10 > http_req_rate(10000)=38788 > > Notice how the http_req_rate keeps going up as does the gpc0 counter yet > wrk doesn't report any failed requests and a result of several thousand > requests per second. > > The impression I get here is that this configuration doesn't *really* > limit the number of requests but only the number of connections based on > the request rate which is semantically a bit different and still allows > a potential abuser to send as many requests as he wants as long as he > keeps using an existing connection. > Is this impressions correct and is the a way to truly limit the number > of requests/s even when no new connections are made? instead of flagging a request, you can simply deny it. HAProxy will then close the TCP connection and the user won't be allowed to establish a new one. > Question 3: As you can see in the configuration I'm using a https > frontend that proxies the traffic to the http frontend so that I can get > the combined stats in the single-process http frontend while still being > able to put the https frontend on independent processes to distribute > the load among cores. > > What I noticed though is that when I do the above tests on the SSL > frontend I don't get any stick-table entries in the regular http > frontend. Apparently the proxied connection aren't registered by the > stick-table. Is there a way to get these connections to show up as well > or do I have to copy+paste the stick-table and abuse settings and keep > them manually in sync between the two frontends? There should be no difference between SSL and clear traffic. I can reproduce the behavior: there might a bug when passing through a unix socket. As a workaround, you can failover to a loopback IP address. In order to populate a blacklist between clear and SSL frontends, you can use the 'http-response add-acl'. Hope this helps. Baptiste
