Hi David, On Wed, Jun 06, 2012 at 01:03:05PM -0700, David Birdsong wrote: > it got me thinking how great it would be if we could bury links in the > manual (or a copy of it) to public urls to these discussions.
It's not worth it I think, and links on the net are short-lived nowadays. Better have some application notes explaining how to do X or Y (which basically is the goal of the quite old arch manual). > it > doesn't fit to have the conversational tone in a manual, but to link > out to this discussion would be so awesome. When you reread the doc in 5 years and find all the links are broken, you'll be a bit disgusted I think :-) > maybe this what stackexchange could be used for, but it'd require us > to copy and replay the whole discussion. No, ideally if we had enough time to spare on documentation, we'd spend less words on the list because things would be much more obvious from the beginning. The problem is that the team working on the code is small and it's more or less the same team who tries to maintain the doc up to date. This is fine when we achieve a stable state, but doing so considerably slows down development phases, which is why doc is always late. > > If you would prevent the backend from queueing any connections, you would > > constantly emit 503 to some visitors, because traffic is never perfectly > > smooth (which is observed by your backend queue inflating). > > i my case, i have another backend fully capable of serving the > requests, so if haproxy can route away accurately, the requests could > land in a capable backend. OK but if you need persistence, you'll break it every time you send a request to the wrong backend. Queues are something normal. For instance, when you look at the traffic on your ethernet link, let's say you're seeing 200 Mbps out. In fact, you're having only bursts of 1 Gbps and pauses of 0. It's thanks to the system's queues that you don't drop packets all the time. Here it's the same. When you run at 3kreq/s, you can be sure that you have peaks of more than 10k/s for a few ms, and pause of several tens to hundreds of ms. You clearly need queues to ensure this does not become a random session killer. > is the 503 only because haproxy can't know ahead of time before > choosing the backend what will happen in the future and so enforching > no queue is technically not reasonable? No, it's simply because haproxy tries applies the load balancing algorithm in the farm, it finds no server (since they're all full), so it wants to put the request in the backend's queue. And if we apply the maxqueue method, then once the backend's queue reaches this value, the request has to be rejected because there's nowhere to service it. With the ACLs at least you can take decisions before reaching this situation. If you have a backend which is never full, you can route the requests there once the first one is considered full, but as explained above, if you don't share the same servers and cookies, you break persistence. > > The best way I found to manage queues is the avg_queue ACL. It returns > > the average number of queued connections per active server. This means > > that wherever the connections are queued, they're accounted for, and > > divided by the number of servers. The result is an average queue length > > for the backend, which translates in an average wait time for visitors. > > cool, i'll use it. is it expensive to calc on every request? I have just rechecked (acl_fetch_avg_queue_size()) and it happens that I had already got rid of the computation loop and used a "totpend" variable instead, to store the global number of queued requests per backend, all servers included. So in fact it's not expensive anymore, it's just a divide by the number of enabled servers. > > BTW, what I'd suggest if you use this is to split the traffic depending > > on the persistence cookie : stop sending new visitors to a backend which > > has too many queued connections, but still send existing ones there. You > > will see that the load on the servers will then regulate itself very well > > and become extremely smooth. And that way you can finely control your > > response times. In short this looks like this : > > > > in this case, requests are completely stateless. we have a pool of 40 > api servers capable of handling paths at /v2/*. the api servers are a > java jetty app. OK. > i'm experimenting with carving up the urls space to > send the same requests to the same servers to see if garbage > collection is slightly more efficient if the servers don't see all of > the api namespace and can 'specialize'. It should theorically be better because the overall code and data should remain in warm cache instead of thrashing all the time. Now how much better it should be is a mystery! Cheers, Willy
