On 4/12/2015 9:16 AM, Phil Steitz wrote: > Can you post your full pool config and info on versions of DBCP and > POOL? > > Do you call isClosed() before closing a connection (not recommended > - that is really the issue in the ticket you commented)? > > To make sure there is not an actual deadlock somewhere, you should > examine a full thread dump. The thread you posted in the ticket is > waiting on the pool, not holding any DBCP or POOL related locks; but > it does hold this application lock: > at com.REDACTED.idxbuild.solr.Chain.doReinsert(Chain.java:1332) > - locked <0x00000000db0011c8> (a java.lang.Object) > Make sure no other threads in process of doing things that will > return connections to the pool aren't waiting on this lock. > > Given that you are using DBCP 2.1, if you have access to a JMX > console it would also be good to look at what is happening in the > pool when your application hangs. In particular, numActive, numIdle > and listAllObjects would be good to look at. Looks like you are > using PoolingDataSource directly, so you need to look at the > underlying GenericObject's JMX properties.
My code does not contain the string "isClosed" at all. I thought I had saved the full thread dump, but the stdout logfile was wiped when I restarted the program, and I didn't think to save a copy. I will be sure to save it next time there's a problem. Here is the entire java file for my "Database" class, edited to redact the company top-level domain: http://apaste.info/6pX The object that I am locking on (0x00000000db0011c8) is a global lock for my whole program which I synchronize on for a handful of normally-short-lived db-related operations just to be sure that they aren't happening simultaneously, and to help make sure that the outcome of certain operations is visible across threads. I'm reasonably sure that there are no operations in those synchronization blocks that are expected to be slow, though I haven't verified that to 100% certainty. One of the operations handled in that kind of synchronization lock is the optimization of certain DB tables, which only happens if the size of those tables is below a certain threshold -- part of making sure that the operation completes quickly and doesn't deadlock the program. Here are all the jar files in my program's lib directory. My program uses javamail, jrobin, dbcp, and solrj: commons-dbcp2-2.1.jar commons-io-2.4.jar commons-pool2-2.3.jar httpclient-4.4.1.jar httpcore-4.4.1.jar httpmime-4.4.1.jar javax.mail.jar jcl-over-slf4j-1.7.12.jar jrobin-1.5.14.jar jul-to-slf4j-1.7.12.jar log4j-1.2.17.jar mysql-connector-java-5.1.35-bin.jar noggit-0.7.jar slf4j-api-1.7.12.jar slf4j-log4j12-1.7.12.jar solr-solrj-5.0.0.jar stax2-api-3.1.1.jar woodstox-core-asl-4.2.0.jar zookeeper-3.4.6.jar I do not have remote JMX enabled for this application, and the server where it runs does not have a gui. I will add some code that spits out the three requested pool stats every time the watchdog thread detects a problem. I cannot predict when there will be a problem ... this code can run for weeks with nothing going wrong, or the deadlock might happen only a few minutes after startup. Thanks, Shawn --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
