2008/6/20 Tobia Conforto <[EMAIL PROTECTED]>: > For the record, this was caused by a deadlock problem, where many > simultaneous requests would get a connection for their SQLTransformer, > exhausting the connection pool, and then get stuck requesting more > connections through an input module I wrote myself, before the > SQLTransformers had a chance to return the connections to the pool. > > I worked around the problem by uncapping the database connections: > > <pool-controller min="100" max="150" max-strict="false"/> > > Now I'd like to know how to avoid the deadlock condition in the first place, > with capped connections, but I'm not sure it can be done. I've asked the > dev list.
Instead of max-strict="false" have you tried blocking="false"? I believe that'll cause it to throw an exception when the pool runs out rather than waiting till another one comes available (which is the reason it deadlocks). Although you probably don't want the users to get a "system error" page, it's better than the whole app locking up. Also try adding a timeout value (e.g. timeout="500"), which I think means that even if it's blocking, it won't do so indefinitely. Not that you'll necessarily notice, as the SQLTransformer tries again a number of times after a delay (both configurable), and by default it'll be 30 seconds or more before it finally gives up. We had a similar problem, just using multiple SQLTransformer components (no input module in our case) in a single pipeline. We fixed it by changing the transformer so that it returned its connection to the pool sooner (in the endDocument method, if I remember correctly, or maybe it was the endElement for the top-level execute-query element) instead of the recycle method. The components in a pipeline are all created (or got from their respective pools) when the pipeline is being built, before it executes, and it only releases/recycles them all once the pipeline finishes executing, so the existing code meant that one pipeline would grab multiple database connections from the pool but not return any of them until the whole thing was done. This meant a handful of simultaneous calls to a pipeline(s) with a few SQLTransformers could deadlock each other (actually, under a high enough load just having two SQLTransformers in a pipeline would be enough) as the transformers later on in the pipeline all wait for a connection (stopping the pipelines from completing) that isn't going to become available as the earlier transformers are still keeping hold of them all... By closing the connections as soon as they're finished with, they become available again for the later transformers to use. I can't remember if I raised a bug/enhancement request in JIRA to change the transformer's behaviour in this way; if I didn't, perhaps you'd like to... Andy. -- http://pseudoq.sourceforge.net/ Open source java Sudoku solver --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
