On 27 Jan 2012, at 05:32, gnath <gautam_exquis...@yahoo.com> wrote: > Hello Chris, > > > After seeing the initial connection pool issue, i started searching online > for help and i found this article : > http://vigilbose.blogspot.com/2009/03/apache-commons-dbcp-and-tomcat-jdbc.html > so, i thought may be tomcat's jar would bring some improvement. by the way, > we had commons-dbcp-1.3.jar. Do you recommend upgrading to newer commons dbcp > jar instead of using tomcat-jdbc.jar.
Tomcat ships with a DBCP implementing of its own. How and where are you defining the database? p > > Just because we are running tomcat-6.0.35, it did not come with > tomcat-jdbc.jar, so we downloaded the 1.1.1 version or jar and dropped in > WEB-INF/lib and started using it. > > > I agree what you are saying about leaking the connection and will plan to set > the logAbandoned flag as you suggested. > > > However, i was about to file a new issue but would like to describe here as > well. So we have 2 servers running tomcat (same code, same configuration). > After we replaced tomcat-jdbc.jar and added 'removeAbandoned' flag to true, > one of the servers is doing great (ofcourse i agree that pool is cleaning up > the mess), but we saw one new issue on the second server. it hasn't been > releasing the connections and was consistently growing slowly. So i collected > thread dump and i saw a deadlock : > > Found one Java-level deadlock: > ============================= > "catalina-exec-1": > waiting to lock monitor 0x000000005d7944b8 (object 0x00000005bd522568, a > com.mysql.jdbc.Connection), > which is held by "[Pool-Cleaner]:Tomcat Connection Pool[1-1015483951]" > "[Pool-Cleaner]:Tomcat Connection Pool[1-1015483951]": > waiting to lock monitor 0x000000005dcdea28 (object 0x00000005bd659ce8, a > com.mysql.jdbc.ResultSet), > which is held by "catalina-exec-1" > > Java stack information for the threads listed above: > =================================================== > "catalina-exec-1": > at com.mysql.jdbc.Connection.getCharsetConverter(Connection.java:3177) > - waiting to lock <0x00000005bd522568> (a com.mysql.jdbc.Connection) > at com.mysql.jdbc.Field.getStringFromBytes(Field.java:583) > at com.mysql.jdbc.Field.getName(Field.java:487) > at com.mysql.jdbc.ResultSet.buildIndexMapping(ResultSet.java:593) > at com.mysql.jdbc.ResultSet.findColumn(ResultSet.java:926) > - locked <0x00000005bd659ce8> (a com.mysql.jdbc.ResultSet) > at com.mysql.jdbc.ResultSet.getInt(ResultSet.java:2401) > > "[Pool-Cleaner]:Tomcat Connection Pool[1-1015483951]": > at com.mysql.jdbc.ResultSet.close(ResultSet.java:736) > - waiting to lock <0x00000005bd659ce8> (a com.mysql.jdbc.ResultSet) > at com.mysql.jdbc.Statement.realClose(Statement.java:1606) > - locked <0x00000005bd522568> (a com.mysql.jdbc.Connection) > - locked <0x00000005bd5e81c0> (a > com.mysql.jdbc.ServerPreparedStatement) > at > com.mysql.jdbc.PreparedStatement.realClose(PreparedStatement.java:1703) > at > com.mysql.jdbc.ServerPreparedStatement.realClose(ServerPreparedStatement.java:901) > - locked <0x00000005bd525ba0> (a java.lang.Object) > - locked <0x00000005bd522568> (a com.mysql.jdbc.Connection) > - locked <0x00000005bd5e81c0> (a > com.mysql.jdbc.ServerPreparedStatement) > at > com.mysql.jdbc.Connection.closeAllOpenStatements(Connection.java:2126) > at com.mysql.jdbc.Connection.realClose(Connection.java:4422) > at com.mysql.jdbc.Connection.close(Connection.java:2098) > - locked <0x00000005bd522568> (a com.mysql.jdbc.Connection) > at > org.apache.tomcat.jdbc.pool.PooledConnection.disconnect(PooledConnection.java:320) > > > > Please help us on this. Could it be a problem with tomcat-jdbc.jar? > > Thanks > -G > > > ________________________________ > From: Christopher Schultz <ch...@christopherschultz.net> > To: Tomcat Users List <users@tomcat.apache.org> > Sent: Thursday, January 26, 2012 9:41 AM > Subject: Re: Tomcat 6.0.35-SocketException: Too many open files issue with > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > G, > > On 1/25/12 11:53 PM, gnath wrote: >> As you have suggested, i started collecting the thread dumps > > Thread dumps will set you free. Well, not really. Instead, they will > tell you where your webapp is breaking, which usually means more work > for you. But at least the hard part is done: finding out what's breaking. > >> when it happened again and we saw some kind of DBCP Connection pool >> issues leading to 'Too Many open files' issue. > > That will definitely do it. > >> So we decided to replace the commons DBCP with tomcat-jdbc.jar >> (with same configuration properties). > > Why? > >> After this change, it seemed for few hours but started seeing in >> the logs where the Connection Pool jar could not give any >> connections and seems to be all the connections are busy. So we >> went ahead and added a configuration property >> 'removeAbandoned=true' in our Datasource configuration. > > I would go back to DBCP unless you think you need to switch for some > reason. > > I suspect you are leaking database connections and don't have a > suitable timeout for removal of "lost" database connections (or maybe > didn't have that set up in the first place). > > You really need to enable "logAbandoned" so you can find out where > your connection leaks are, and fix them. In development, set > maxActive="1" and leave it there, forever. Also, set > logAbandoned="true" and always run like that in development. Running > like that in production isn't a bad idea, either. > >> We are still watching the performance and the server behavior >> after these changes. Will keep you posted on how things will turn >> out or if i see any further issues. > > I suspect you are still leaking connections, but your pool is now > silently cleaning-up after the mess your webapp is making. Instrument > your pool. Fix your leaks. > > - -chris > -----BEGIN PGP SIGNATURE----- > Version: GnuPG/MacGPG2 v2.0.17 (Darwin) > Comment: GPGTools - http://gpgtools.org > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEARECAAYFAk8hkDkACgkQ9CaO5/Lv0PCxFgCgs+EiV/CNjmCNekeDwKHgnNtZ > 5LYAoKZUkIAJOK0eItkoHBF3wScK9lQf > =AyL4 > -----END PGP SIGNATURE----- > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org