Oleg, I've implemented the IdleConnectionEvictor you suggested, and whilst it seems to do the job (demonstrated by watching the TCP connections in Wireshark), I'm seeing some curious entries in the log:
16-Mar-2009 14:25:24 org.apache.http.impl.conn.IdleConnectionHandler remove WARNING: Removing a connection that never existed! This is being caused by the line: connMgr.closeIdleConnections(30, TimeUnit.SECONDS); This suggests to me that I'm doing something wrong here, but I'm at a loss as to what. Any suggestions would be most appreciated. Thanks, Sam 2009/3/2 Sam Crawford <[email protected]> > Perfect, that should do the job nicely. > > Many thanks, > > Sam > > > 2009/3/2 Oleg Kalnichevski <[email protected]> > > On Sun, 2009-03-01 at 22:34 +0000, Sam Crawford wrote: >> > Hello, >> > >> > I'm having an issue with HttpClient communicating reliably with >> webservers >> > behind a load balancer. In summary, the first request goes through fine >> and >> > the load balancer closes the server>client connection 30 seconds later, >> > leaving the TCP connection half open. This same connection is then >> accessed >> > again much later as the client tries to close its half and a timeout >> occurs >> > (as the load balancer forgot about it ages ago). The full flow is as >> > follows: >> > >> > 1. Client makes a request to balancer1.com (for example). A new TCP >> > connection is established (fresh three way handshake). >> > 2. Client issues GET request for some content, and all goes well. >> > 3. 30 seconds later, the server (balancer1.com) sends a FIN/ACK, and >> the >> > client dutifully responds with an ACK. (note: at this point the >> > server>client connection is closed, but the client>server connection is >> > still open) >> > 4. A long time later (e.g. 4000 seconds in my testing) the client is >> asked >> > to make another request to balancer1.com >> > 5. The client host sends a FIN/ACK on the same connection as was >> established >> > in step #1. No response is received (as the load balancer has timed out >> this >> > connection presumably), and it retries the FIN/ACK for 30 seconds. >> > 6. The client gives up and establishes a new TCP connection and requests >> the >> > content, and all is well. >> > >> > The issue with the above is the approx 30 second delay that occurs as >> the >> > client is attempting to close a connection that has long since been >> > forgotten about by the other party. I realise that load balancer >> dropping >> > the connection is causing the problem, but I'm sure there must be a way >> I >> > can set the HttpClient to actively send a follow-up FIN/ACK if one is >> > received from the other end? Ideally I still want to have persistent >> > HTTP/1.1 connections enabled to this server, as there will be busy peak >> > periods and very quiet periods. >> > >> >> Sam, >> >> This actually a limitation of the classic (blocking) I/O model in >> general. Blocking sockets cannot react to any I/O events unless they are >> read from or written to by a thread. Persistent connections are kept in >> the pool inactive (detached from execution thread), so they are not able >> to detect changes in the connection state and react to them >> appropriately. >> >> The only way around this problem is running a connection eviction thread >> that wakes up at a regular interval and drops expired / idle connections >> from the connection pool. See sample below: >> >> >> http://svn.apache.org/viewvc/httpcomponents/httpclient/trunk/httpclient/src/examples/org/apache/http/examples/client/ClientEvictExpiredConnections.java?view=markup >> >> Hope this helps >> >> Oleg >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> >
