On Wed, Aug 08, 2012 at 12:51:23AM -0600, Andrew Davidoff wrote:
> Willy,
>
> Thanks for the quick response. I haven't fully digested your example
> suggestion yet but I will sit down with it and the haproxy configuration
> documentation and sort it out in my brain.
>
> Here's the basic idea of the use case. Let me go ahead and state that maybe
> haproxy just isn't the right solution here. There are many ways to solve
> this, It just seemed to me like haproxy might have been a magic answer.
>
> We make a bunch of requests to an API that rate limits based on source IP.
> To maximize our overall request rate, we utilize proxies to afford us more
> source IPs. Even if those proxies can handle a ton of work themselves, if
> we push them, individually, over the API's rate limits, they can be
> temporarily or permanently disallowed from accessing the API.
OK I see now. It's not a performance limit but a contractual limit you
must not go over if you don't want to be banned.
Why don't you use a larger number of IP addresses to access your API then ?
> Right now our API clients (scripts) handle rate limiting themselves. The
> way they currently do this involves knowledge of the per-source-IP rate
> limit for the API they're talking to, and how many proxies live behind a
> squid instance that all their requests go through. That squid instance
> hands out proxies round-robin, which is what makes the request rate work.
>
> Based on how the scripts currently handle the rate limiting, we start
> running into problems if we want multiple scripts accessing the same API to
> run at the same time. Basically, each running script must then know about
> any other scripts that are running and talking to the same API, so it can
> adjust its request rate accordingly, and anything already running needs be
> informed that more scripts access the same API have started up, so it can
> do the same.
>
> Additionally, we run into the problem of proxies failing. If a proxy fails
> and the scripts don't learn then and adjust their rate limits, then the
> per-proxy rate limit has inadvertently increased across all proxies.
These are precisely the types of issues that you won't have with the
two-stage LB I gave in the example, because the load sent to the servers
will be smoothed as much as possible and will never go beyond the configured
limit. In my tests I have always observed 3-4 digits stability in rate-limits.
The first stage will ensure a global distribution and the second stage will
ensure you never go over the limit for each server.
You can even use this to increase the number of source addresses, with all
servers going to the same address (some people use this to browse via
multiple outgoing ADSL lines). For instance, let's say your API is on
192.168.0.1:80 and you want to use 3 different source IP addresses
(192.168.0.11..13) each limited to exactly 10 requests per second, and
you want to distribute the load across them in order to always use them
to the max possible rate :
listen front
bind :80
balance leastconn
server srv1 127.0.0.1:8000 maxconn 100 track back1/srv
server srv2 127.0.0.2:8000 maxconn 100 track back2/srv
server srv3 127.0.0.3:8000 maxconn 100 track back3/srv
listen back1
bind 127.0.0.1:8000
rate-limit 10
server srv 192.168.0.1:80 source 192.168.0.11 check
listen back2
bind 127.0.0.2:8000
rate-limit 10
server srv 192.168.0.1:80 source 192.168.0.12 check
listen back3
bind 127.0.0.3:8000
rate-limit 10
server srv 192.168.0.1:80 source 192.168.0.13 check
Regards,
Willy
