Re: Tomcat connection pool bleeding under heavy load

2013-11-21 Thread marko lugarič
Hello I guess you are suggesting to compile this class with the changes you propose and test it? regards, marko On Thu, Nov 21, 2013 at 1:58 AM, David Bullock david.bull...@machaira.com.au wrote: Oh, and 'throw e;' at the end of the handler :-) David Bullock Machaira Enterprises Pty Ltd

Re: Tomcat connection pool bleeding under heavy load

2013-11-21 Thread marko lugarič
Hi I guess i should code something like start i.e. 50 (configurable parameter to increase the load) threads and issues queries non stop over jdbc to i.e. 2 database test tables - we should generate enough trasactions to take all pool connections. And I should probably use jdbc connections as

Re: Tomcat connection pool bleeding under heavy load

2013-11-20 Thread marko lugarič
Hello The tests are run in the following manner: around 5 min to start the server and connect all clients. Then I can apply load from clients: on the graph I applied only half load for first seven minutes an then full load but this is not required - I can go with full load after all clients

Re: Tomcat connection pool bleeding under heavy load

2013-11-20 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Marko, On 11/20/13, 8:14 AM, marko lugarič wrote: Hello The tests are run in the following manner: around 5 min to start the server and connect all clients. Then I can apply load from clients: on the graph I applied only half load for first

Re: Tomcat connection pool bleeding under heavy load

2013-11-20 Thread marko lugarič
Hello I have repeated the test with logAbandoned set to true and left it half hour after pool is empty. There are a lot of exceptions (Pool empty. Unable to fetch a connection in 15 seconds and Pool wait interrupted) but i don't know what exactly to search (search with Abandoned did not produce

Re: Tomcat connection pool bleeding under heavy load

2013-11-20 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Marko, On 11/20/13, 12:07 PM, marko lugarič wrote: I have repeated the test with logAbandoned set to true and left it half hour after pool is empty. There are a lot of exceptions (Pool empty. Unable to fetch a connection in 15 seconds and Pool

Re: Tomcat connection pool bleeding under heavy load

2013-11-20 Thread David Bullock
Line 150 of org.apache.tomcat.jdbc.pool.FairBlockingQueue can throw InterruptedException, which should be handled by: catch (InterruptedException e) { lock.lock(); waiters.remove(c); // prevent future threads from offering to this thread, since we are about to die lock.unlock(); if

Re: Tomcat connection pool bleeding under heavy load

2013-11-20 Thread David Bullock
Oh, and 'throw e;' at the end of the handler :-) David Bullock Machaira Enterprises Pty Ltd PO Box 31 Canowindra NSW 2804 02 6344 1100 http://machaira.com.au/ On 21 November 2013 11:48, David Bullock david.bull...@machaira.com.au wrote: Line 150 of

Re: Tomcat connection pool bleeding under heavy load

2013-11-19 Thread David Bullock
On 19 November 2013 14:59, Caldarale, Charles R chuck.caldar...@unisys.com wrote: From: David Bullock [mailto:david.bull...@machaira.com.au] Subject: Re: Tomcat connection pool bleeding under heavy load In PooledConnection#borrow(int,String,String) when handling InterruptedException, the code

Re: Tomcat connection pool bleeding under heavy load

2013-11-19 Thread marko lugarič
Hello About using useEquals=false property I guess we defined when we started using this pool. After reading the documentation i guess it is better to leave it out (it has no effect on the test - i tested it). We dont have long running queries because there is not much data and all operations

Re: Tomcat connection pool bleeding under heavy load

2013-11-19 Thread David Bullock
Marko wrote: In that case debugging looks like the only option? Nobody yet disputed: The 'leakiness' of the pool is actually in org.apache.tomcat.jdbc.pool.FairBlockingQueue#poll() If the line (!c.await(timeout, unit)) exits via an InterruptedException, the the 'exchangecountdownlatch'

Re: Tomcat connection pool bleeding under heavy load

2013-11-19 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Marko, On 11/19/13, 10:16 AM, marko lugarič wrote: About using useEquals=false property I guess we defined when we started using this pool. After reading the documentation i guess it is better to leave it out (it has no effect on the test - i

Tomcat connection pool bleeding under heavy load

2013-11-18 Thread marko lugarič
Hello! We have received an issue from our customer that after some time in production they get empty connection pool. After some time we managed to reproduce their problem in our test environment with smaller number of pool connections. So generally we have a problem that if connection pool is

Re: Tomcat connection pool bleeding under heavy load

2013-11-18 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Marko, On 11/18/13, 3:48 AM, marko lugarič wrote: We have received an issue from our customer that after some time in production they get empty connection pool. After some time we managed to reproduce their problem in our test environment with

Re: Tomcat connection pool bleeding under heavy load

2013-11-18 Thread David Bullock
In PooledConnection#borrow(int,String,String) when handling InterruptedException, the code there does: Thread.interrupted();//clear the flag, and bail out but if the comment is correct, then the code is wrong (should be Thread.isInterrupted()). However, the entire line is probably not

RE: Tomcat connection pool bleeding under heavy load

2013-11-18 Thread Caldarale, Charles R
From: David Bullock [mailto:david.bull...@machaira.com.au] Subject: Re: Tomcat connection pool bleeding under heavy load In PooledConnection#borrow(int,String,String) when handling InterruptedException, the code there does: Thread.interrupted();//clear the flag, and bail out