Re: Tomcat 8 DB Connection Pooling

2015-06-12 Thread Douglas Schaible
Hi Chris,

I have the connection pool defined in  /etc/tomcat8/context.xml.  Is that the 
wrong place the define it?  Is what is defined there being copied into each 
application?  I underplayed some sample application and I am now seeing 10 time 
the number of apps being deployed DB connections.

I am creating war files from eclipse to be deployed.  Should I be creating a 
context file in eclipse so that it is included in the war file?  If yes, should 
I copy the context.xml from from /etc/tomcat8 and then remove the pool 
definition from that file and just have it in the one in the war file?

Responses to the questions are below.

I have also implemented what Filip recommended, however after 24 hours the app 
start intermittently failing and there are only 2 open connections to the DB.

This is in the log:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet 
successfully received from the server was 79,290,702 milliseconds ago.  The 
last packet sent successfully to the server was 79,290,702 milliseconds ago. is 
longer than the server configured value of 'wait_timeout'. You should consider 
either expiring and/or testing connection validity before use in your 
application, increasing the server configured values for client timeouts, or 
using the Connector/J connection property 'autoReconnect=true' to avoid this 
problem.
12-Jun-2015 16:51:55.659 WARNING [Tomcat JDBC Pool 
Cleaner[147643626:1434048550658]] 
org.apache.tomcat.jdbc.pool.ConnectionPool.abandon Connection has been 
abandoned 
PooledConnection[com.mysql.jdbc.JDBC4Connection@1fa3cd0e]:java.lang.Exception


79,290,702 milliseconds is ~22 hours if my calculations are correct.  That is 
way way longer than what is specified.

Here is what I added:
logAbandoned=true
removeAbandoned=true
removeAbandonedTimeout=60 
timeBetweenEvictionRunsMillis=15000 



Thanks,
Doug


 On Jun 12, 2015, at 4:07 PM, Christopher Schultz 
 ch...@christopherschultz.net wrote:
 
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA256
 
 Douglas,
 
 In addition to what Filip posted...
 
 On 6/11/15 12:18 PM, Douglas Schaible wrote:
 Good Day All,
 
 I am having a problem with a connection pool and I was hoping for 
 some guidance.
 
 I have defined the connection pool below for two deployed 
 applications to use. When I bounce the server I can see that it 
 immediately crates 100 connections to the DB. (I am ok with this,
 but that is not what is specified in the config).
 
 When you start your application, does it load a bunch of stuff from
 the database? If you are fetching lots of connections and not
 returning them, you may see your connection count go up to 100 (the
 maxTotal value).
 

No, it does not.  Both of the apps on the server only make connections to the 
DB in doGet and they do not have an init method.


 Then when I access the application repeatedly very quickly new 
 connections to the DB are created each time a page is called till
 it reaches about 150 and then future calls fail with the message 
 com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException:
 
 
 
 Too many connections” logged by the server.
 
 I would have expected the pool to stop handing-out connections at 100.
 
 But, if you don't have removeAbandoned and you aren't returning the
 connections properly, then the pool is full of connections that can't
 be used.

The server will make more than 100 connections, it will make about 150 and then 
start giving the error.





 
 If we don’t place any extra load on the server the number of 
 connections to the DB are reduced to 1 or 2 after 24 hours and you 
 get the error Timeout: Pool empty. Unable to fetch a connection
 in 30 seconds, none available[size:100; busy:100; idle:0; 
 lastwait:3].” from one of the applications and the other 
 application continues to work just fine.
 
 The other application likely is using a different pool. Where is that
 Resource defined?

it uses the same pool.  The pool is defined in /etc/tomcat8/context.xml.  There 
is only one Resource


 
 I believe that the DB, MySQL, is dropping the connection, but
 tomcat does not know that it has been dropped.
 
 I am looking for some guidance on how to tune this configuration. 
 Any suggestions?
 
 
 Resource name=jdbc/PDB auth=Container 
 type=javax.sql.DataSource 
 factory=org.apache.tomcat.jdbc.pool.DataSourceFactory 
 initialSize=10 maxTotal=100 maxIdle=5 maxWaitMillis=1 
 username=“xx password=“xx 
 driverClassName=com.mysql.jdbc.Driver 
 url=jdbc:mysql://x.xx.us-east-1.rds.amazonaws.com”/
 
 Your error message says Unable to fetch a connection in 30 seconds,
 but your maxWaitMillis is 1 (10 seconds). Are you sure this is the
 configuration being used?

Yes.  I am not sure why the timeout is different or more connections are made 
than expected.  If I removed the configuration from context.xml and bounce the 
server all of the connection attempts the DB fail, so I believe that config is 
being used

Tomcat 8 DB Connection Pooling

2015-06-11 Thread Douglas Schaible
Good Day All,

I am having a problem with a connection pool and I was hoping for some guidance.

I have defined the connection pool below for two deployed applications to use.  
When I bounce the server I can see that it immediately crates 100 connections 
to the DB.  (I am ok with this, but that is not what is specified in the 
config)  Then when I access the application repeatedly very quickly new 
connections to the DB are created each time a page is called till it reaches 
about 150 and then future calls fail with the message 
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Too many 
connections” logged by the server. 

If we don’t place any extra load on the server the number of connections to the 
DB are reduced to 1 or 2 after 24 hours and you get the error Timeout: Pool 
empty. Unable to fetch a connection in 30 seconds, none available[size:100; 
busy:100; idle:0; lastwait:3].” from one of the applications and the other 
application continues to work just fine. 

I believe that the DB, MySQL, is dropping the connection, but tomcat does not 
know that it has been dropped. 

I am looking for some guidance on how to tune this configuration.  Any 
suggestions? 



 Resource
   name=jdbc/PDB
   auth=Container
   type=javax.sql.DataSource
   factory=org.apache.tomcat.jdbc.pool.DataSourceFactory
   initialSize=10
   maxTotal=100
   maxIdle=5
   maxWaitMillis=1
   username=“xx
   password=“xx
   driverClassName=com.mysql.jdbc.Driver
   url=jdbc:mysql://x.xx.us-east-1.rds.amazonaws.com”/


Server version:Apache Tomcat/8.0.20
Server number: 8.0.20.0
OS Name:   Linux
OS Version:3.14.35-28.38.amzn1.x86_64
Architecture:  amd64
JVM Version:   1.7.0_79-mockbuild_2015_04_22_23_18-b00
JVM Vendor:Oracle Corporation

Thanks,
Doug