Hello Bill, A connection leak, interesting. Maybe we have a similar issue.
It's a bit tricky since we are using iBatis which is doing a lot of things behind the scenes, but I think we could also implement such a logging wrapper around the pool. However it seems that our pool exhaustion is caused by a combination of poor database performance and high concurrency; the webapp is using Ajax and there can be many (concurrent?) database requests caused by a single user action. Thanks for sharing your experience, I'll get back with our findings! Torsten -----Original Message----- From: Wm.A.Stafford [mailto:[EMAIL PROTECTED] Sent: 4. december 2007 19:56 To: Jakarta Commons Users List Subject: Re: DBCP Locking Up? Torsten, We were seeing the same behavior in a lightly loaded application when the connection pool was exhausted. I spent several days trying to figure out what was happening. A google search turned up several forum and mailing list postings reporting the same problem. I eventually came to the conclusion that the deadlock is some sort of undocumented behavior of DBCP that occurs when the pool can no longer fulfill a request for a connection. This was with dbcp-1.2.2 and commons-pool-1.3. I was pretty sure our pool exhaustion was caused by a connection leak because we used dbcp to replace a non-pooling DataSource. So connections which would have formerly been garbage collected if not closed now were leaked. We located the leaks by wrapping DBCP with class that logged the state of the pool for every open and close, ex: log.info(" active connections=" + pool.getNumActive() + " idle connections=" + pool.getNumIdle()) ; Eliminating the connection leak eliminated our deadlocks but I never figured out what the underlying DBCP problem is. -=bill [EMAIL PROTECTED] wrote: > We are using DBCP with iBatis in a JSF/Ajax webapp running in Tomcat with an > Oracle database. > > The application has quite high load and concrrency, and the database is very > loaded. The application is running fine for some days but then freezes at > peak hours. > > When I look at a thread dump made while the app hangs, I can see as many > threads as we have connections in the pool (64) in BLOCKED state while > calling either borrowObject() or returnObject(), please see the two examples > below. It looks like some kind of deadlock and the only way out of it seems > to be a restart of Tomcat. > > First I thought this is normal with the WHEN_EXHAUSTED_LOCK action, but since > we have maxWait set to 1000 I'd not expect this kind of blocking to happen. > > I tried to reproduce the behavior with a small test app, but here the threads > go in TIMED_WAIT state when the pool is exhausted and become RUNNABLE again > as soon as some connection became available again. > > I really don't understand what is happening and why the pool gets locked up > that way. > > Any hint/suggestion welcome! > > Torsten > > "TP-Processor483" daemon prio=10 tid=0x01fe5000 nid=0x11f7 waiting for > monitor entry [0x8d87e000..0x8d881788] > java.lang.Thread.State: BLOCKED (on object monitor) > at > org.apache.commons.pool.impl.GenericObjectPool.returnObject(GenericObjectPool.java:916) > - waiting to lock <0x9ddf7460> (a > org.apache.commons.pool.impl.GenericObjectPool) > at > org.apache.commons.dbcp.PoolableConnection.close(PoolableConnection.java:87) > - locked <0xf83f9eb8> (a org.apache.commons.dbcp.PoolableConnection) > at > org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.close(PoolingDataSource.java:181) > at > com.ibatis.sqlmap.engine.transaction.jdbc.JdbcTransaction.close(JdbcTransaction.java:81) > at > com.ibatis.sqlmap.engine.transaction.TransactionManager.end(TransactionManager.java:110) > > "TP-Processor382" daemon prio=10 tid=0x026f6000 nid=0x25a waiting for monitor > entry [0x7de7f000..0x7de81808] > java.lang.Thread.State: BLOCKED (on object monitor) > at > org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:781) > - waiting to lock <0x9ddf7460> (a > org.apache.commons.pool.impl.GenericObjectPool) > at > org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:96) > at > org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:880) > at > com.ibatis.sqlmap.engine.transaction.jdbc.JdbcTransaction.init(JdbcTransaction.java:48) > at > com.ibatis.sqlmap.engine.transaction.jdbc.JdbcTransaction.getConnection(JdbcTransaction.java:89) > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > -- Wm.A.Stafford Principal Engineer The Institute of Marine and Coastal Sciences Rutgers University 732-932-6555 x367 [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
