Just enabled "option http-keep-alive" on my personal websites to enable HTTP keepalives from client to servers :)
Baptiste On Mon, Dec 16, 2013 at 10:50 AM, Arnall <[email protected]> wrote: > Great news Willy, thanks a lot for all of this, and thanks to all the > contributors ! > > Le 16/12/2013 03:41, Willy Tarreau a écrit : > >> Hi all, >> >> here is probably the largest update we ever had, it's composed of 345 >> patches! >> >> Some very difficult changes had to be made and as usual when such changes >> happen, they take a lot of time due to the multiple attempts at getting >> them right, and as time goes, people submit features :-) >> >> After two weeks spent doing only fixes, I thought it was time to issue >> dev20. >> I'm sure I'll forget a large number of things, but the main features of >> this >> version include the following points (in merge order) : >> >> - optimizations (splicing, polling, etc...) : a few percent CPU could >> be >> saved ; >> >> - memory : the connections and applets are now allocated only when >> needed. >> Additionally, some structures were reorganized to avoid fragmentation >> on >> 64-bit systems. In practice, an idle session size has dropped from >> 1936 >> bytes to 1296 bytes (-640 bytes, or -33%). >> >> - samples : all sample fetch expressions now support a comma-delimited >> list of converters. This is also true in ACLs, so that it becomes >> possible to do things like : >> >> # convert to lower case and use fast tree indexing >> acl known_domain hdr(host),lower -f huge-domain-list.lst >> >> - a lot of code has been deduplicated in the tracked counters, it's now >> possible to use sc_foo_bar(1, args) instead of sc1_foo_bar(args). >> Doing >> so has simplified the code and makes life of APIs easier. >> >> - it's now possible to look up a tracked key from another table. This >> allows >> to retrieve multiple counters for the same key. >> >> - several hash algorithms are provided, and it is possible to select >> them >> per backend. This high quality work was done at Tumblr by Bhaskar >> Maddala. >> >> - agent-checks: this new feature was merged and replaced the >> lb-agent-chk. >> Some changes are still planned but feedback is welcome. The goal of >> this >> agent is to retrieve soem weight information from a server >> independantly >> of the service health. A typical usage would consist in reporting the >> server's idle percentage as an estimate of the possible weight. This >> work >> was done by Simon Horman for Loadbalancer.org. >> >> - samples : more automatic conversions between types are supported, >> making >> it easier to stick to any parameter. The types are much more dynamic >> now. >> Some improvements are still pending. This work was done by Thierry >> Fournier >> at Exceliance. >> >> - map : a new type of converter appeared : maps. A map matches a key >> from >> a file just like ACLs do, and replaces this value with the value >> associated >> with the key on the same line of the file. As it is a converter, it >> can be >> used in any sample expression. The first usage consists in >> geolocation, >> where networks are associated with country codes. Maps may be >> consulted, >> deleted, updated and filled from the CLI. Some will probably use this >> to >> program actions or emulate ACLs without even reloading a config. This >> work was also achieved by Thierry Fournier, and reviewed by Cyril >> Bonté >> who developped the original Geoip patchset for 1.4 and 1.5. >> >> - http-request redirect now supports log-format like expressions, just >> like >> http-request add-header. This allows to emit strings extracted from >> the >> request (host header, country code from a map, ...). Thierry again >> here. >> >> - checks: tcp-check supports send/expect sequences with >> strings/regex/binary. >> Thus it now becomes possible to check unsupported protocols, even >> binary. >> This work is from Baptiste Assmann. >> >> - keep-alive: the dynamic allocation of the connection and applet in >> the >> session now allows to reuse or kill a connection that was previously >> associated with the session. Thus we now have a very basic support >> for >> keep-alive to the servers. There is even an option to relax the load >> balancing to try to keep the same connection. Right now we don't do >> any connection sharing so the main use is for static servers and for >> far remote servers or those which require the broken NTLM auth. That >> said, the performance tests I have run show an increase from 71000 >> connections per second to 150000 keep-alive requests per second >> running >> on one core of a Xeon E5 3.6 GHz. This doubled to 300k requests per >> second with two cores. I didn't test above, I lacked injection tools >> :-) >> One good point is that it will help people assemble haproxy and >> varnish >> together with haproxy doing the consistent hash and varnish caching >> after >> it. >> >> As most of you know, server-side keep-alive is the condition to release >> 1.5. >> Now we have it, we'll be able to improve on it but it's basically working. >> >> I expect to release 1.5-final around January and mostly focus on chasing >> bugs till there. So I'd like to set a feature freeze. I know it doesn't >> mean much considering that we won't stop contribs. But I don't want to >> merge another large patch set before the release. Ideally there will not >> be any dev21 version. Reality probably is that we'll have to issue one >> because people will inevitably report annoying bugs that were not reported >> in snapshots. >> >> I still have to perform difficult changes on the health checks system to >> adapt the agent-check after identifying some limitations caused by the >> original design we decided on a few months ago. >> >> Another set of pending changes concerns the polling. Yes I know, each time >> I touch the pollers I break things. But I need to do them, as now with >> keep-alive it becomes obvious that we waste too much time enabling and >> disabling polling because we miss one flag ("this FD last returned >> EAGAIN"). >> The good point is that it will simplify the connection management and >> checks. >> >> If these points are done quick enough, I'll see if I can implement a very >> basic support for server connection sharing connection (without which I >> still consider keep-alive as not a huge improvement). >> >> Note to users of 1.5-dev19 who hesitate to test this one : 71 bugs were >> fixed >> since dev19. Others probably added, but hopefully less. >> >> Usual links below : >> >> Site index : http://haproxy.1wt.eu/ >> Sources : http://haproxy.1wt.eu/download/1.5/src/devel/ >> Changelog : http://haproxy.1wt.eu/download/1.5/src/CHANGELOG >> Cyril's HTML doc : >> http://cbonte.github.com/haproxy-dconv/configuration-1.5.html >> >> I'm not appending the changelog, it's too large. >> >> Have fun and please report issues here, >> Willy >> >> > >
