Re: Keep-Alive timeout & persistent connections

Tue, 01 Jul 2008 16:17:08 -0700

That did it. I'm running svn head on one production server and seeing very good behavior. 

On Jun 30, 2008, at 6:53 PM, Sam Berlin wrote:


Hi Paul,

I think I've fixed this one -- the problem was using deleteEntry
instead of dropEntry.  The former required the entry to be considered
'free', but the entry had just been allocated and was definitely not
'free'.  I'm pretty certain this will indirectly fix the
connection/memory leak -- a side effect of the problem was connection
requests would just pile up and never be serviced.

Sam
On Mon, Jun 30, 2008 at 10:33 AM, Paul Lindner <[EMAIL PROTECTED]> wrote:
I tested this code using production traffic and the results are not so good. Ended up with a connection/memory leak. The jmap histogram of a server in 
this bad state is attached.
Also, I had some issues with one host, it used a keepalive with a timeout=1. It appears that the route pool filled up with connections. It also appears that when it closes an expired free connection that it is not released back into the pool... Here's the debug output showing the final connection allocation and the attempt to get a connection from a full pool. It appears that Freeing the connection puts it back on the Idle pool, but it becomes 
inaccessible for some reason.


2008-06-29 19:22:27,761 [http-80-55] DEBUG
org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager  -
ThreadSafeClientConnManager.getConnection:
HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com], timeout =
50002008-06-29 19:22:27,761 [http-80-55] DEBUG
org.apache.http.impl.conn.tsccm.ConnPoolByRoute - Getting free connection 
[HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
2008-06-29 19:22:27,761 [http-80-55] DEBUG
org.apache.http.impl.conn.tsccm.ConnPoolByRoute - Closing expired free connection [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]] [null] 
2008-06-29 19:22:27,761 [http-80-55] DEBUG
org.apache.http.impl.conn.DefaultClientConnection - Connection closed 
2008-06-29 19:22:27,761 [http-80-55] DEBUG
org.apache.http.impl.conn.tsccm.ConnPoolByRoute - No free connections 
[HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
2008-06-29 19:22:27,761 [http-80-55] DEBUG
org.apache.http.impl.conn.tsccm.ConnPoolByRoute - Available capacity: 1 
[HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
2008-06-29 19:22:27,761 [http-80-55] DEBUG
org.apache.http.impl.conn.tsccm.ConnPoolByRoute - Creating new connection 
[HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]]
2008-06-29 19:22:27,761 [http-80-55] DEBUG
org.apache.http.impl.conn.DefaultClientConnection - Connection closed 
2008-06-29 19:22:27,818 [http-80-55] DEBUG
org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Freeing connection
[HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
2008-06-29 19:22:27,818 [http-80-55] DEBUG
org.apache.http.impl.conn.IdleConnectionHandler - Adding connection at: 
12147925478182008-06-29 19:22:27,818
[http-80-55] DEBUG org.apache.http.impl.conn.tsccm.ConnPoolByRoute  -
Notifying no-one, there are no waiting threads

-----------------------------------------------
2008-06-29 19:22:29,274 [http-80-26] DEBUG
org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager  -
ThreadSafeClientConnManager.getConnection:
HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com], timeout = 5000
2008-06-29 19:22:29,274 [http-80-26] DEBUG
org.apache.http.impl.conn.tsccm.ConnPoolByRoute - Getting free connection 
[HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
2008-06-29 19:22:29,274 [http-80-26] DEBUG
org.apache.http.impl.conn.tsccm.ConnPoolByRoute - Closing expired free connection [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]] [null] 
2008-06-29 19:22:29,274 [http-80-26] DEBUG
org.apache.http.impl.conn.DefaultClientConnection - Connection closed 
2008-06-29 19:22:29,274 [http-80-26] DEBUG
org.apache.http.impl.conn.tsccm.ConnPoolByRoute - No free connections 
[HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
2008-06-29 19:22:29,274 [http-80-26] DEBUG
org.apache.http.impl.conn.tsccm.ConnPoolByRoute - Available capacity: 0 
[HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
2008-06-29 19:22:29,274 [http-80-26] DEBUG
org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Need to wait for
connection [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]] [null] 
2008-06-29 19:22:29,331 [http-80-79] DEBUG
org.apache.http.impl.conn.DefaultClientConnection - Connection shut down 
2008-06-29 19:22:29,331 [http-80-79] DEBUG
org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Freeing connection
[HttpRoute[{}->http://ads.socialmedia.com]][null]
2008-06-29 19:22:29,331 [http-80-79] DEBUG
org.apache.http.impl.conn.tsccm.ConnPoolByRoute - Notifying thread waiting 
on any pool
2008-06-29 19:22:29,331 [http-80-26] DEBUG
org.apache.http.impl.conn.tsccm.ConnPoolByRoute - No free connections 
[HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
2008-06-29 19:22:29,331 [http-80-26] DEBUG
org.apache.http.impl.conn.tsccm.ConnPoolByRoute - Available capacity: 0 
[HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
2008-06-29 19:22:29,331 [http-80-26] DEBUG
org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Need to wait for
connection [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]] [null] 
2008-06-29 19:22:29,331 [http-80-26] DEBUG
org.apache.http.impl.conn.tsccm.ConnPoolByRoute - No free connections 
[HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
2008-06-29 19:22:29,331 [http-80-26] DEBUG
org.apache.http.impl.conn.tsccm.ConnPoolByRoute - Available capacity: 0 
[HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]][null]
2008-06-29 19:22:29,331 [http-80-26] DEBUG
org.apache.http.impl.conn.tsccm.ConnPoolByRoute  - Need to wait for
connection [HttpRoute[{}->http://ayi.h5.snapinteractiveapps.com]] [null] 



Size    Count   Class description
-------------------------------------------------------
684164112       558913  byte[]
492584016       2134710 char[]
209551728 4365661 java.util.concurrent.locks.ReentrantLock $NonfairSync 209517696 4364952 java.util.concurrent.ConcurrentHashMap $Segment 144470200 4364956 java.util.concurrent.ConcurrentHashMap $HashEntry[] 
101379880       2534497 java.lang.String
65693112        628849  java.lang.Object[]
47067200        588340  com.friend.data.UserAppBean
41467120 272810 java.util.concurrent.ConcurrentHashMap $Segment[] 
37081896        272661  org.apache.catalina.session.StandardSession
32830608        683971  java.util.HashMap$Entry
30965200        275343  java.util.Hashtable$Entry[]
24135760        603394  java.util.ArrayList
22612816        115641  java.util.HashMap$Entry[]
19871936        620998  java.util.Date
19642320        272810  java.util.concurrent.ConcurrentHashMap
18644208        224156  int[]
17608256        275129  java.util.Hashtable
14569296 303527 java.util.concurrent.ConcurrentHashMap $HashEntry 
13089600        272700  java.beans.PropertyChangeSupport
13054072        97229   * ConstMethodKlass
12340480        154256  org.apache.tomcat.util.buf.MessageBytes
11679960        97229   * MethodKlass
10593856        165529  org.apache.tomcat.util.buf.ByteChunk
10529104        10652   * ConstantPoolKlass




---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]




---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Paul Lindner
[EMAIL PROTECTED]




---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Reply via email to