Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Mark, On 5/17/13 5:36 AM, Mark Thomas wrote: On 17/05/2013 09:28, Michael-O wrote: Hi folks, there's now a follow-up on the issue [1]. Recap: JreMemoryLeakPreventionListener reported that my webapp has spawned OracleTimeoutPollingThread because I have set a QueryTimeout in the JDBC Pool. Mark Thomas noted that this is likely a bug in the driver. I have taken action and created a service request with Oracle. My personal analysis with VisualVM: The thread is spawned when the first query is run. The thread keeps running when the webapp is shut down or undeployed. Classes remain in memory. The JDBC Pool does a simple Class.forName to load the driver, it neither uses the DriverManager loading nor does it actively unload the driver. Oracle answer: They were able to reproduce the issue. The technical analysis says: Hi Michael, I confirmned internally that this message from Tomcat can be ignored as there is no risk of any real leak from OracleTimeoutPollingThread thread. This thread is related to the JDBC driver which may be used by many apps simultaneously. Unloading the app does not unload the driver and should not and does not stop the thread. It seems to be the design behaviour. So my questions would be: 1. Is that still not a false positive? No, that is not a false positive. The response from Oracle is wrong. There is nothing wrong with the driver creating the thread or the thread continuing after the web application has stopped. The problem is as follows: 1. When the JDBC driver creates the Thread, the Thread's context class loader (as returned by Thread.getContextClassLoader()) is set to the web application's class loader. 2. The correct behaviour at this point would be for the Driver to set the Thread's context class loader to the class loader that loaded the Driver class when the Thread is created. I have recently observed with MySQL Connector/J that setting the context ClassLoader to something else (in their case, null) is not sufficient[1]. The problem is that the Thread's inherited, cached ProtectionDomain chain is still linked-up with the WebappClassLoader and so switching the contextClassLoader after thread-creation is not effective. I think the only way to do it is by having the code that creates the Thread make a contextClassLoader switch first, then create the Thread, then switch back. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJRr5CuAAoJEBzwKT+lPKRY1OMQAJ6oQ4XjVzXq2imrG52HDix/ E1XNQYJGbjNIs+zS/JL8qjcKWDYGpykYobIZ5gNOCe5I+W419DSTOW9ff8/9djQF Ui1IeLHe+NAUITxbqgrQjnxED0kkrBwrPDJLbr/+l4nW5ljKDkF9nv0EV+qDgx1L cJ9ahr4FwsLo7NiHZ/4GjQ8T6gSyp3T2EpqCAE7gqDjYPyYT8i+GPLgxVHSFNzQy 8IYiqe7T7gPQwv8xrYQMks0L9HfnEdMrM4E+S1G5fIOP25VODR9BldMU8YvMKhLo GXD+/dy0msl9IWoSKGoJMQmNQanh9uj74whmNQ3Ulf6Z7Jviz6OuHiukuNY7Awke wL5U0J2ExJdFbHw8Jc67TvhmoIkmQ8N/nNI99RvZ5uVIHQ9dsg5hbRuc0V3E8ZJp JSAVjqEUhHTC7E7DWwBqknAX4F1herCCeIBDJhh0HMJ+MYRWgSZTk/4ccEhEjkU8 1Yj7YFBaWucWGF4SPtac6YjItlWzGYNjVG5jKqkX1cYcIlfIJtuvPGy9nWDDl0Lv pn1yWoirokwGssv5YEhUxeWTFnJoHD08ii17PJSkwBzL3l84ZVmIZbrodA+xWTNo oTqVxjxSL0uOB7pI7TmGVWzm8GKLChoiHTwE88jt7KD1wLkRdwMDXdrCVlaZKAjq pHTL8jry5JtriwOyQHhW =sQg+ -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Chuck, On 5/22/13 1:42 PM, Caldarale, Charles R wrote: From: Christopher Schultz [mailto:ch...@christopherschultz.net] Subject: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread I suspect that the DriverManager will always be loaded by the boot ClassLoader, since the default-dispatch for ClassLoaders is to chekc the parent first, then check yourself. The DriverManager is at the top-level (well, there is primordial, but that doesn't really count) ClassLoader so you'll always get that. Terminology alert: you're confusing the boot class loader with the system class loader, and have erroneously labeled the actual boot class loader as primordial. Generally speaking, the boot class loader is responsible for extracting JRE classes from rt.jar (and others that come with the JRE), while the system class loader deals with those specified by the java.class.path setting (CLASSPATH for those still stuck on environment variables). Thanks for the pedantry: I was in fact ignoring the difference between the system and boot ClassLoaders. However, the primordial ClassLoader does in fact exist, and does in fact load classes, and is not called the boot ClassLoader. Oracle describes the primordial ClassLoader as that which loads pretty much everything in the java.* package space. Oracle also describes the system ClassLoader as the delegation root ancestor of all class loaders.[1] In practice, the primordial class loader is identified by a null reference (and is this difficult to inspect) and the system class loader is represented by an ExtClassLoader. On top of that is an AppClassLoader. I'll have to play around with some classes loaded via endorsed directories to see if I can nail-down how to get a Class with the ExtClassLoader as its class loader instead of AppClassLoader. DriverManager's ClassLoader is in fact null, the primordial class loader. You can test it yourself, and discover the ClassLoader hierarchy in play with a simple program: public class ClassLoaderTest { public static void main(String[] args) { printClassLoaders(ClassLoaderTest.class); printClassLoaders(Thread.class); printClassLoaders(Object.class); printClassLoaders(java.sql.DriverManager.class); } public static void printClassLoaders(Class cls) { ClassLoader cl = cls.getClassLoader(); System.out.println(*** ClassLoaders for + cls.getName() + ***); while(null != cl) { System.out.println(ClassLoader[ + cls.getName() + ]: + cl); cl = cl.getParent(); } System.out.println(ClassLoader[ + cls.getName() + ]: + cl); } } Run under two different JVMs on my laptop, I get: $ java -showversion ClassLoaderTest java version 1.7.0_17 Java(TM) SE Runtime Environment (build 1.7.0_17-b02) Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode) *** ClassLoaders for ClassLoaderTest *** ClassLoader[ClassLoaderTest]: sun.misc.Launcher$AppClassLoader@63644028 ClassLoader[ClassLoaderTest]: sun.misc.Launcher$ExtClassLoader@4ab03512 ClassLoader[ClassLoaderTest]: null *** ClassLoaders for java.lang.Thread *** ClassLoader[java.lang.Thread]: null *** ClassLoaders for java.lang.Object *** ClassLoader[java.lang.Object]: null *** ClassLoaders for java.sql.DriverManager *** ClassLoader[java.sql.DriverManager]: null $ java ClassLoaderTest java version 1.6.0_45 Java(TM) SE Runtime Environment (build 1.6.0_45-b06-451-11M4406) Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01-451, mixed mode) *** ClassLoaders for ClassLoaderTest *** ClassLoader[ClassLoaderTest]: sun.misc.Launcher$AppClassLoader@a6eb38a ClassLoader[ClassLoaderTest]: sun.misc.Launcher$ExtClassLoader@69cd2e5f ClassLoader[ClassLoaderTest]: null *** ClassLoaders for java.lang.Thread *** ClassLoader[java.lang.Thread]: null *** ClassLoaders for java.lang.Object *** ClassLoader[java.lang.Object]: null *** ClassLoaders for java.sql.DriverManager *** ClassLoader[java.sql.DriverManager]: null $ java -showversion ClassLoaderTest java version 1.6.0_45 Java(TM) SE Runtime Environment (build 1.6.0_45-b06-451-11M4406) Java HotSpot(TM) Client VM (build 20.45-b01-451, mixed mode) *** ClassLoaders for ClassLoaderTest *** ClassLoader[ClassLoaderTest]: sun.misc.Launcher$AppClassLoader@cf2c80 ClassLoader[ClassLoaderTest]: sun.misc.Launcher$ExtClassLoader@1729854 ClassLoader[ClassLoaderTest]: null *** ClassLoaders for java.lang.Thread *** ClassLoader[java.lang.Thread]: null *** ClassLoaders for java.lang.Object *** ClassLoader[java.lang.Object]: null *** ClassLoaders for java.sql.DriverManager *** ClassLoader[java.sql.DriverManager]: null - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org
Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 All, On 5/23/13 10:40 AM, Christopher Schultz wrote: Chuck, On 5/22/13 1:42 PM, Caldarale, Charles R wrote: From: Christopher Schultz [mailto:ch...@christopherschultz.net] Subject: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread I suspect that the DriverManager will always be loaded by the boot ClassLoader, since the default-dispatch for ClassLoaders is to chekc the parent first, then check yourself. The DriverManager is at the top-level (well, there is primordial, but that doesn't really count) ClassLoader so you'll always get that. Terminology alert: you're confusing the boot class loader with the system class loader, and have erroneously labeled the actual boot class loader as primordial. Generally speaking, the boot class loader is responsible for extracting JRE classes from rt.jar (and others that come with the JRE), while the system class loader deals with those specified by the java.class.path setting (CLASSPATH for those still stuck on environment variables). Thanks for the pedantry: I was in fact ignoring the difference between the system and boot ClassLoaders. However, the primordial ClassLoader does in fact exist, and does in fact load classes, and is not called the boot ClassLoader. Oracle describes the primordial ClassLoader as that which loads pretty much everything in the java.* package space. Oracle also describes the system ClassLoader as the delegation root ancestor of all class loaders.[1] In practice, the primordial class loader is identified by a null reference (and is this difficult to inspect) and the system class loader is represented by an ExtClassLoader. On top of that is an AppClassLoader. I'll have to play around with some classes loaded via endorsed directories to see if I can nail-down how to get a Class with the ExtClassLoader as its class loader instead of AppClassLoader. DriverManager's ClassLoader is in fact null, the primordial class loader. You can test it yourself, and discover the ClassLoader hierarchy in play with a simple program: public class ClassLoaderTest { public static void main(String[] args) { printClassLoaders(ClassLoaderTest.class); printClassLoaders(Thread.class); printClassLoaders(Object.class); printClassLoaders(java.sql.DriverManager.class); } public static void printClassLoaders(Class cls) { ClassLoader cl = cls.getClassLoader(); System.out.println(*** ClassLoaders for + cls.getName() + ***); while(null != cl) { System.out.println(ClassLoader[ + cls.getName() + ]: + cl); cl = cl.getParent(); } System.out.println(ClassLoader[ + cls.getName() + ]: + cl); } } Run under two different JVMs on my laptop, I get: $ java -showversion ClassLoaderTest java version 1.7.0_17 Java(TM) SE Runtime Environment (build 1.7.0_17-b02) Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode) *** ClassLoaders for ClassLoaderTest *** ClassLoader[ClassLoaderTest]: sun.misc.Launcher$AppClassLoader@63644028 ClassLoader[ClassLoaderTest]: sun.misc.Launcher$ExtClassLoader@4ab03512 ClassLoader[ClassLoaderTest]: null *** ClassLoaders for java.lang.Thread *** ClassLoader[java.lang.Thread]: null *** ClassLoaders for java.lang.Object *** ClassLoader[java.lang.Object]: null *** ClassLoaders for java.sql.DriverManager *** ClassLoader[java.sql.DriverManager]: null $ java ClassLoaderTest java version 1.6.0_45 Java(TM) SE Runtime Environment (build 1.6.0_45-b06-451-11M4406) Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01-451, mixed mode) *** ClassLoaders for ClassLoaderTest *** ClassLoader[ClassLoaderTest]: sun.misc.Launcher$AppClassLoader@a6eb38a ClassLoader[ClassLoaderTest]: sun.misc.Launcher$ExtClassLoader@69cd2e5f ClassLoader[ClassLoaderTest]: null *** ClassLoaders for java.lang.Thread *** ClassLoader[java.lang.Thread]: null *** ClassLoaders for java.lang.Object *** ClassLoader[java.lang.Object]: null *** ClassLoaders for java.sql.DriverManager *** ClassLoader[java.sql.DriverManager]: null $ java -showversion ClassLoaderTest java version 1.6.0_45 Java(TM) SE Runtime Environment (build 1.6.0_45-b06-451-11M4406) Java HotSpot(TM) Client VM (build 20.45-b01-451, mixed mode) *** ClassLoaders for ClassLoaderTest *** ClassLoader[ClassLoaderTest]: sun.misc.Launcher$AppClassLoader@cf2c80 ClassLoader[ClassLoaderTest]: sun.misc.Launcher$ExtClassLoader@1729854 ClassLoader[ClassLoaderTest]: null *** ClassLoaders for java.lang.Thread *** ClassLoader[java.lang.Thread]: null *** ClassLoaders for java.lang.Object *** ClassLoader[java.lang.Object]: null *** ClassLoaders for java.sql.DriverManager *** ClassLoader[java.sql.DriverManager]: null -chris Forgot my reference: http://docs.oracle.com/javase/6/docs/technotes/guides/security/spec/security-spec.doc5.html -BEGIN PGP SIGNATURE- Version: GnuPG
RE: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
From: Christopher Schultz [mailto:ch...@christopherschultz.net] Subject: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread Thanks for the pedantry: I was in fact ignoring the difference between the system and boot ClassLoaders. However, the primordial ClassLoader does in fact exist, and does in fact load classes, and is not called the boot ClassLoader. What you're calling the primordial class loader _is_ the boot class loader, identified by a null reference. (The use of primordial in the page you referred to is unusual; it's known as the boot or bootstrap class loader in almost all other documentation.) It's responsible for more than just the java.* packages, since it also loads all the com.sun.*, sun.*, and other JVM-vendor supplied classes. I ignored the extensions class loader, since it's rarely used and was not pertinent to the topic. the system class loader is represented by an ExtClassLoader. On top of that is an AppClassLoader. What you're looking at is JVM-internal classes (hence the $ in the names), that just happen to be the ones chosen in current versions of the JVM. The name is an implementation detail; the internal class mechanism is used to handle privilege issues. The internal names should not be construed as descriptive of the class loader hierarchy. On top of that is an AppClassLoader. Only for programs that supply their own, such as Tomcat. My comments concerned the JVM itself, not Tomcat. DriverManager's ClassLoader is in fact null, the primordial class loader. Unless one configures a replacement DriverManager (I think there's a system property for that, but I'm not sure). - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
Am 2013-05-21 20:38, schrieb Mark Thomas: On 21/05/2013 19:01, Michael-O wrote: Mark, I did receive an answer to the issue, citing your findings. See verbatim copy below: Hi Michael, I received the following update from our developer: ** The theoretical problem is that the thread is holding the app class loader so it remains reachable and the app is never unloaded. So if the user loads and unloads the app, the app classes remain in memory. Subsequent loading and unloading of the app will not get pinned in memory as the thread is already running so the subsequent loading and unloading will not cause additional class loaders to be pinned. It is a fixed size memory leak. It does not grow. The thread suggests setting the context class loader to be the parent of the default context class loader. This may work in Tomcat but it's pretty random. I am researching the problem to determine what if anything the driver should do to work across all containers. A Tomcat specific fix is not acceptable. Almost but not quite. The correct fix is to set the context class loader of the Thread to the class loader that loaded the Driver or, alternatively, the class loader that loaded the thread class. As Mark Thomas pointed out it is critical that the driver is loaded in the boot or system or container class loader, not the app class loader. If the driver is in the app class loader then when the app is unloaded the driver also should be unloaded. Unfortunately this doesn't work. The driver itself remains reachable so the app class loader is reachable so the app is never unloaded. At present there is no general way to solve this problem. The necessary hooks are added in JDK 8. I'd agree to this point. Most users put the driver in the container, not the app, so this is rarely a problem. I don't agree with this. I often see this with JDBC drivers which is why Tomcat has a pile of code specifically to unpick the mess this creates. ** I will certainly have to fill out a bug to have it investigated officially. That is good news. Seems like they understood the problem. But I do doubt that this is a fixed size moemory leak. I think the point they are trying to make is that it is only the first instance of the web application to be unloaded that will be pinned in memory. Subsequent reloads won't trigger a leak. That is correct. Mark, here's the next update from Oracle on the issue: 1. For those with Oracle Support access, here's the bug id: 16841748 2. Quote: I don't think any system property is needed. We have a solution for the timeout thread that should work for all app containers. So Oracle is working on a fix, my SR remains open until I receive an updated ojdbc.jar. Michael - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Chuck, On 5/23/13 12:01 PM, Caldarale, Charles R wrote: From: Christopher Schultz [mailto:ch...@christopherschultz.net] Subject: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread Thanks for the pedantry: I was in fact ignoring the difference between the system and boot ClassLoaders. However, the primordial ClassLoader does in fact exist, and does in fact load classes, and is not called the boot ClassLoader. What you're calling the primordial class loader _is_ the boot class loader, identified by a null reference. (The use of primordial in the page you referred to is unusual; it's known as the boot or bootstrap class loader in almost all other documentation.) It's responsible for more than just the java.* packages, since it also loads all the com.sun.*, sun.*, and other JVM-vendor supplied classes. I ignored the extensions class loader, since it's rarely used and was not pertinent to the topic. While I haven't exactly implemented my own JVM or anything like that, I have often heard the boot class loader referred to as the primordial class loader. Either way, we're talking about the same thing. the system class loader is represented by an ExtClassLoader. On top of that is an AppClassLoader. What you're looking at is JVM-internal classes (hence the $ in the names), that just happen to be the ones chosen in current versions of the JVM. The name is an implementation detail; the internal class mechanism is used to handle privilege issues. The internal names should not be construed as descriptive of the class loader hierarchy. I wasn't inferring anything from the names of the ClassLoader objects: just showing what the hierarchy was given the code I wrote. On top of that is an AppClassLoader. Only for programs that supply their own, such as Tomcat. My comments concerned the JVM itself, not Tomcat. Er, I meant that the AppClassLoader (an Oracle-specific thing) was in the hierarchy. The hierarchy I was showing was what actually happens when I run that example class from the command-line, so Tomcat isn't involved. Oracle's JVM has 2 ClassLoader objects between my class and the boot class loader. Presumably one of them is for loading extensions (ExtClassLoader) and then the other (Oracle's AppClassLoader) loads stuff from the CLASSPATH (or -classpath or java.class.path). When placing a JAR file into one of the paths found in java.ext.dirs, the parent ClassLoader for those classes ends up being Oracle's ExtClassLoader -- just underneath the ClassLoader that manages the items on the java.class.path. When I modified java.endorsed.dirs, the classes loaded from those directories were loaded by the primordial/boot class loader. Interesting. I would have expected a layer between them, but I guess it makes sense as you have to be able to override things like org.xml.*, etc. So, at least in Oracle's JVM: java.endorsed.dirs - boot class loader (null) java.ext.dirs - another class loader (ExtClassLoader) java.class.path- a third class loader (AppClassLoader) (In that order: most user classes will start at the bottom and search upwards.) - From within Tomcat, of course, there will be a longer chain from webapp-loaded classes up to the top. DriverManager's ClassLoader is in fact null, the primordial class loader. Unless one configures a replacement DriverManager (I think there's a system property for that, but I'm not sure). +1 ...but who uses DriverManager anymore, anyway? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJRnpSbAAoJEBzwKT+lPKRYoP0QALrFCAJuCVfCP4BHNLhyGu3F 7YvT1IFS7zSUZlKDf44vOqRfgvQoUdU+Lpj/LTjUxVTEzbvF+5W3LmmQ7Efo3bJd PA1Td2pqGJkUbcO7R1aSF2Xlaqzz9VzKvfjakenBq4MRzBCkzv06eNOoB+tEnyG1 uq+4CB5t4qS1kdEyZAI9wEPLK7wvHJFWZUk8s/NqK/X+rHA1yhzxWjWCOO+EjjZl 0TsQt44aVWhuL8X+goumQRfDkVjI0lpgOnhRQvnZo3b81H9zhhqvOzD9/JucOf+r PwuSRIOLlLloCq1sCSNXgG0YT4lkJsReYmXM8mMjSY8EEa7aTeqZQzLfM1AOLNI4 Wk+fCwHDBxpnBP70oKISVRvEFVgvn4nQIDK9JxVikT/p+QhY6CZU7vSzZMnTDPHP rXaJ2orhp4SIgb1Pr0VZ5RdLn785iEyxzqHVdONik+3XK/1azQ6JKe9TEEXXgVXS AFuuajDm2yJhsYFeYmsKbgAO5L30jxU3/YhgQ0toyYvPi8n11w4J4hjn9gB6WQ2s Rw+3nQ+25Jcww5sZ08nJdu+vcK/tWvjz6/B/vGrkpAilRZs5xxKAmwb4V+G5NiBs Fr3ph5Ys5kV03axHZRRxhjxqfF/GZdeAVHDc57//Vd0S9zuyZ1CWmXVqAJXfrnil fafc/jc6sAnd865xQ23X =w6CF -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
From: Christopher Schultz [mailto:ch...@christopherschultz.net] Subject: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread While I haven't exactly implemented my own JVM or anything like that, I have... I have often heard the boot class loader referred to as the primordial class loader. Both the JVM and Java Language specs refer to it as the bootstrap class loader: http://docs.oracle.com/javase/specs/jvms/se7/html/jvms-5.html#jvms-5.3.1 The word primordial does not appear anywhere in the JVM spec, and only as a descriptive term for the Object class in the language spec. So, at least in Oracle's JVM: Also in any JVM serious about running Java programs. java.endorsed.dirs - boot class loader (null) The boot[strap] class loader, as noted. java.ext.dirs - another class loader (ExtClassLoader) The extensions class loader, rarely used. java.class.path- a third class loader (AppClassLoader) The system class loader, officially (java.lang.ClassLoader.getSystemClassLoader()). Interestingly enough, the above hierarchy does not appear in any spec I can find, although you'd be hard pressed to find a JVM that doesn't implement it that way. Using AppClassLoader as the internal name for the system class loader is unfortunate, since it creates confusion with common usage and an actual Application Class Loader, as described in this link: http://docs.oracle.com/cd/E19501-01/819-3659/beadf/index.html That one does seem like a bit of overkill... - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Mark, On 5/21/13 2:38 PM, Mark Thomas wrote: On 21/05/2013 19:01, Michael-O wrote: Mark, I did receive an answer to the issue, citing your findings. See verbatim copy below: Hi Michael, I received the following update from our developer: ** The theoretical problem is that the thread is holding the app class loader so it remains reachable and the app is never unloaded. So if the user loads and unloads the app, the app classes remain in memory. Subsequent loading and unloading of the app will not get pinned in memory as the thread is already running so the subsequent loading and unloading will not cause additional class loaders to be pinned. It is a fixed size memory leak. It does not grow. The thread suggests setting the context class loader to be the parent of the default context class loader. This may work in Tomcat but it's pretty random. I am researching the problem to determine what if anything the driver should do to work across all containers. A Tomcat specific fix is not acceptable. Almost but not quite. The correct fix is to set the context class loader of the Thread to the class loader that loaded the Driver +1 or, alternatively, the class loader that loaded the thread class. Do you mean java.lang.Thread (primordial)? Or whatever subclass they are using? Maybe it would be more accurate to say the ClassLoader of the runnable being run which covers thread-subclasses or standard Runnables being run by a standard Thread. As Mark Thomas pointed out it is critical that the driver is loaded in the boot or system or container class loader, not the app class loader. If the driver is in the app class loader then when the app is unloaded the driver also should be unloaded. Unfortunately this doesn't work. The driver itself remains reachable so the app class loader is reachable so the app is never unloaded. At present there is no general way to solve this problem. The necessary hooks are added in JDK 8. I'd agree to this point. Most users put the driver in the container, not the app, so this is rarely a problem. I don't agree with this. I often see this with JDBC drivers which is why Tomcat has a pile of code specifically to unpick the mess this creates. They are missing the point: the leak cannot be avoided, even if the driver is (correctly) placed in the container-level ClassLoader and not the application. It's frustrating when driver authors take so long so understand what is going on and how easy (and reasonable) it is to fix the problem. ** I will certainly have to fill out a bug to have it investigated officially. That is good news. Seems like they understood the problem. But I do doubt that this is a fixed size moemory leak. I think the point they are trying to make is that it is only the first instance of the web application to be unloaded that will be pinned in memory. Subsequent reloads won't trigger a leak. That is correct. That is little solace to folks who have more Class objects than just about anything else. With a huge number of support libraries being used to operate a fairly lean transaction-processing system, a single reload could bust the heap and/or PermGen. Just because it's a fixed-size memory leak doesn't mean it isn't worth fixing. :( - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJRnNGtAAoJEBzwKT+lPKRY6GUP/02KynMptUNCIZQp1ZafsfeY ugNyicVw9BJ8Nw7tmW2vyRvFFNUDeXpZfS6iD5vSNU3QnSg/Sr5qs3oHSQHltjw8 VsBwVwc2Co6wHwR2FrW2LQda3TNHke5XCWmVVvJi+clJvl4zQ8LWINO+hmzOXgPL O29hStrvkRhT5F5GBMnu9AIdFR8eccgv/BuLmQ3ASnoG7YRRiLSKkFFJuYzbKnFK VoJwaJBK5QAbjqi/wAWF3mCOgqvzrozuV7RrrJ9Ah/5J74VQoBdIuvPG7oYDXu8n bZBEIEhLrHvc6wUevXKR3K6xFGQxI+0+Nv5GmqjuehktHx/Cf3Td7c0hIPNw0sgN VSASIhG0//x35e7lhzroeLjSUgvQNfcYH6d7JKropvkNQOsxV42ISVMVQouKLMJ+ fPEwy1ZyoUrJ6XCrzozyePaL5K18BlL8OGGs5yo5UWxoBonLvh20cJjGAkAh/Er2 a1Dwijc2ulkO4WnJn3RpKaCuEYCC8lBwakqAwLdzC5nJVQD5ZaQPcK8P63Zd5DeM Kmkli3NEILi8JAaVHIoYB6fmd4VBH4sbGJbccISX65SyVr4HpnUbPcLgjBxlL0PG RV8MwZ5fGmxpbVrXBbK7sO/a951kib+GJiMU8wlAqs0+cDc5kse/YzOEl+lc8iG1 krw5V+wCqB/NR8e+nTGQ =lN0r -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
On May 22, 2013, at 9:09 AM, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Mark, On 5/21/13 2:38 PM, Mark Thomas wrote: On 21/05/2013 19:01, Michael-O wrote: Mark, I did receive an answer to the issue, citing your findings. See verbatim copy below: Hi Michael, I received the following update from our developer: ** The theoretical problem is that the thread is holding the app class loader so it remains reachable and the app is never unloaded. So if the user loads and unloads the app, the app classes remain in memory. Subsequent loading and unloading of the app will not get pinned in memory as the thread is already running so the subsequent loading and unloading will not cause additional class loaders to be pinned. It is a fixed size memory leak. It does not grow. The thread suggests setting the context class loader to be the parent of the default context class loader. This may work in Tomcat but it's pretty random. I am researching the problem to determine what if anything the driver should do to work across all containers. A Tomcat specific fix is not acceptable. Almost but not quite. The correct fix is to set the context class loader of the Thread to the class loader that loaded the Driver +1 or, alternatively, the class loader that loaded the thread class. Do you mean java.lang.Thread (primordial)? Or whatever subclass they are using? Maybe it would be more accurate to say the ClassLoader of the runnable being run which covers thread-subclasses or standard Runnables being run by a standard Thread. Or why not just java.sql.DriverManager.class.getClassLoader()? Since drivers are always stored in the DriverManager regardless of what class loader they're loaded in (hence why you shouldn't put the driver in WEB-INF/lib), isn't the alway-correct solution to use the class loader that loaded the DriverManager for this thread? As Mark Thomas pointed out it is critical that the driver is loaded in the boot or system or container class loader, not the app class loader. If the driver is in the app class loader then when the app is unloaded the driver also should be unloaded. Unfortunately this doesn't work. The driver itself remains reachable so the app class loader is reachable so the app is never unloaded. At present there is no general way to solve this problem. The necessary hooks are added in JDK 8. I'd agree to this point. Most users put the driver in the container, not the app, so this is rarely a problem. I don't agree with this. I often see this with JDBC drivers which is why Tomcat has a pile of code specifically to unpick the mess this creates. They are missing the point: the leak cannot be avoided, even if the driver is (correctly) placed in the container-level ClassLoader and not the application. It's frustrating when driver authors take so long so understand what is going on and how easy (and reasonable) it is to fix the problem. ** I will certainly have to fill out a bug to have it investigated officially. That is good news. Seems like they understood the problem. But I do doubt that this is a fixed size moemory leak. I think the point they are trying to make is that it is only the first instance of the web application to be unloaded that will be pinned in memory. Subsequent reloads won't trigger a leak. That is correct. That is little solace to folks who have more Class objects than just about anything else. With a huge number of support libraries being used to operate a fairly lean transaction-processing system, a single reload could bust the heap and/or PermGen. Just because it's a fixed-size memory leak doesn't mean it isn't worth fixing. :( Agreed. Nick - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Nick, On 5/22/13 10:18 AM, Nick Williams wrote: On May 22, 2013, at 9:09 AM, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Mark, On 5/21/13 2:38 PM, Mark Thomas wrote: On 21/05/2013 19:01, Michael-O wrote: Mark, I did receive an answer to the issue, citing your findings. See verbatim copy below: Hi Michael, I received the following update from our developer: ** The theoretical problem is that the thread is holding the app class loader so it remains reachable and the app is never unloaded. So if the user loads and unloads the app, the app classes remain in memory. Subsequent loading and unloading of the app will not get pinned in memory as the thread is already running so the subsequent loading and unloading will not cause additional class loaders to be pinned. It is a fixed size memory leak. It does not grow. The thread suggests setting the context class loader to be the parent of the default context class loader. This may work in Tomcat but it's pretty random. I am researching the problem to determine what if anything the driver should do to work across all containers. A Tomcat specific fix is not acceptable. Almost but not quite. The correct fix is to set the context class loader of the Thread to the class loader that loaded the Driver +1 or, alternatively, the class loader that loaded the thread class. Do you mean java.lang.Thread (primordial)? Or whatever subclass they are using? Maybe it would be more accurate to say the ClassLoader of the runnable being run which covers thread-subclasses or standard Runnables being run by a standard Thread. Or why not just java.sql.DriverManager.class.getClassLoader()? Since drivers are always stored in the DriverManager regardless of what class loader they're loaded in (hence why you shouldn't put the driver in WEB-INF/lib), isn't the alway-correct solution to use the class loader that loaded the DriverManager for this thread? I suspect that the DriverManager will always be loaded by the boot ClassLoader, since the default-dispatch for ClassLoaders is to chekc the parent first, then check yourself. The DriverManager is at the top-level (well, there is primordial, but that doesn't really count) ClassLoader so you'll always get that. I don't think that will work because the boot ClassLoader won't be able to load the JDBC driver's classes, since they are in the container's ClassLoader. (boot ClassLoader should be set to CATLAINA_BASE/bin/*.jar while the container's ClassLoader will be set to CATALINA_BASE/lib/*.jar). Also remember that not all JDBC drivers are registering drivers: some were explicitly-written *not* to register themselves because of foolishness like this. When you are using a connection-pool, there's no reason to register the driver with the DriverManager because the pool itself acts kind of as the driver manager. For example MySQL's Connector/J com.mysql.jdbc.Driver class extends NonRegisteringDriver and adds only a static initializer that basically does this: static { DriverManager.register(new Driver()); } So if I wanted to (and I probably should!), I could change my driver configuration from com.mysql.jdbc.Driver to com.mysql.jdbc.NonRegisteringDriver and avoid the DriverManager entirely. (And your suggestion above would not really be valid anymore, since the DriverManager is not involved). - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJRnPPXAAoJEBzwKT+lPKRYLV8QAMIlybE7cMnT2XQTbmi9bB5+ RZqS+VHTCf6q36senH0KP5DF1b6d1XMcJEfkpGAOUsA7jye50NFbUf5HICOKqX2D PSOt2GlDISHQvTyG3U2sBBZyxmtfHW9y13t+397qhC2X806WhbJGPWyWID9vCqX0 EcVF02jwTq6aWpI/aKasxuwS3BDpm7EUMloBX7XdHUYO8lR562rA68KWxGZO5CqD x9Jho7wtUgoj2RlIB4/7mQQKO1k14upS9OFgqVbkLYRA2RLhT5eOHZTBT4jeIOYJ gcNnF2f3hEx6rXeNaEdapn46wv1h4dVUNy4mvPi1THb0qb82NEsPYNIadV7CRDAT eIjzLcjruab5FKlMMq8wRppXh7/+Y21Bd4f3xaPB2b9rmiuv0a4ixnpWGg4QH/3C NZpKM1F+Lv4jkRaF45Je+Pqna3d/WTgN5kMwwVmC0q0f1Dab1nR0FmGBayRGat+/ qHodLD8bWZ2EVHBvmFXr3BHa4HAsNFSx6Wi630EOHuzFyNHUbkLP2MZxJiufM3+h 9uSDyDuoqWY16EauwHsnkouRAA8O56NlgJ8EALyO/+KV7mOFdjrT/O2VTYWIyVrf sY3Kn+1UtLXEGf42p9K9fjdG/sGMD4PgJPryokg7UkyoxzP2mkWzOL4Rd1YtWzDD 0RraqDL/r9LszWrPilmN =QT5q -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
On May 22, 2013, at 11:35 AM, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Nick, On 5/22/13 10:18 AM, Nick Williams wrote: On May 22, 2013, at 9:09 AM, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Mark, On 5/21/13 2:38 PM, Mark Thomas wrote: On 21/05/2013 19:01, Michael-O wrote: Mark, I did receive an answer to the issue, citing your findings. See verbatim copy below: Hi Michael, I received the following update from our developer: ** The theoretical problem is that the thread is holding the app class loader so it remains reachable and the app is never unloaded. So if the user loads and unloads the app, the app classes remain in memory. Subsequent loading and unloading of the app will not get pinned in memory as the thread is already running so the subsequent loading and unloading will not cause additional class loaders to be pinned. It is a fixed size memory leak. It does not grow. The thread suggests setting the context class loader to be the parent of the default context class loader. This may work in Tomcat but it's pretty random. I am researching the problem to determine what if anything the driver should do to work across all containers. A Tomcat specific fix is not acceptable. Almost but not quite. The correct fix is to set the context class loader of the Thread to the class loader that loaded the Driver +1 or, alternatively, the class loader that loaded the thread class. Do you mean java.lang.Thread (primordial)? Or whatever subclass they are using? Maybe it would be more accurate to say the ClassLoader of the runnable being run which covers thread-subclasses or standard Runnables being run by a standard Thread. Or why not just java.sql.DriverManager.class.getClassLoader()? Since drivers are always stored in the DriverManager regardless of what class loader they're loaded in (hence why you shouldn't put the driver in WEB-INF/lib), isn't the alway-correct solution to use the class loader that loaded the DriverManager for this thread? I suspect that the DriverManager will always be loaded by the boot ClassLoader, since the default-dispatch for ClassLoaders is to chekc the parent first, then check yourself. The DriverManager is at the top-level (well, there is primordial, but that doesn't really count) ClassLoader so you'll always get that. I don't think that will work because the boot ClassLoader won't be able to load the JDBC driver's classes, since they are in the container's ClassLoader. (boot ClassLoader should be set to CATLAINA_BASE/bin/*.jar while the container's ClassLoader will be set to CATALINA_BASE/lib/*.jar). Also remember that not all JDBC drivers are registering drivers: some were explicitly-written *not* to register themselves because of foolishness like this. When you are using a connection-pool, there's no reason to register the driver with the DriverManager because the pool itself acts kind of as the driver manager. For example MySQL's Connector/J com.mysql.jdbc.Driver class extends NonRegisteringDriver and adds only a static initializer that basically does this: static { DriverManager.register(new Driver()); } So if I wanted to (and I probably should!), I could change my driver configuration from com.mysql.jdbc.Driver to com.mysql.jdbc.NonRegisteringDriver and avoid the DriverManager entirely. (And your suggestion above would not really be valid anymore, since the DriverManager is not involved). - -chris Ahh. That makes sense. And I will be changing my context.xml resource definitions now... :-) Nick - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
From: Christopher Schultz [mailto:ch...@christopherschultz.net] Subject: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread I suspect that the DriverManager will always be loaded by the boot ClassLoader, since the default-dispatch for ClassLoaders is to chekc the parent first, then check yourself. The DriverManager is at the top-level (well, there is primordial, but that doesn't really count) ClassLoader so you'll always get that. Terminology alert: you're confusing the boot class loader with the system class loader, and have erroneously labeled the actual boot class loader as primordial. Generally speaking, the boot class loader is responsible for extracting JRE classes from rt.jar (and others that come with the JRE), while the system class loader deals with those specified by the java.class.path setting (CLASSPATH for those still stuck on environment variables). - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
Am 2013-05-17 14:26, schrieb Mark Thomas: On 17/05/2013 12:31, Michael-O wrote: Hi Mark, thanks again for the detailed answer, details inline. Gesendet: Freitag, 17. Mai 2013 um 11:36 Uhr Von: Mark Thomas ma...@apache.org An: Tomcat Users List users@tomcat.apache.org Betreff: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread On 17/05/2013 09:28, Michael-O wrote: Hi folks, there's now a follow-up on the issue [1]. Recap: JreMemoryLeakPreventionListener reported that my webapp has spawned OracleTimeoutPollingThread because I have set a QueryTimeout in the JDBC Pool. Mark Thomas noted that this is likely a bug in the driver. I have taken action and created a service request with Oracle. My personal analysis with VisualVM: The thread is spawned when the first query is run. The thread keeps running when the webapp is shut down or undeployed. Classes remain in memory. The JDBC Pool does a simple Class.forName to load the driver, it neither uses the DriverManager loading nor does it actively unload the driver. Oracle answer: They were able to reproduce the issue. The technical analysis says: Hi Michael, I confirmned internally that this message from Tomcat can be ignored as there is no risk of any real leak from OracleTimeoutPollingThread thread. This thread is related to the JDBC driver which may be used by many apps simultaneously. Unloading the app does not unload the driver and should not and does not stop the thread. It seems to be the design behaviour. So my questions would be: 1. Is that still not a false positive? No, that is not a false positive. The response from Oracle is wrong. There is nothing wrong with the driver creating the thread or the thread continuing after the web application has stopped. The problem is as follows: 1. When the JDBC driver creates the Thread, the Thread's context class loader (as returned by Thread.getContextClassLoader()) is set to the web application's class loader. 2. The correct behaviour at this point would be for the Driver to set the Thread's context class loader to the class loader that loaded the Driver class when the Thread is created. 3. The memory leak occurs as follows: - the web application is stopped - Tomcat clears all references to the web application and its classes - The web application should be eligible for garbage collection - The JDBC driver is still loaded (as it should be) - The JDBC driver retains a reference to the Thread (as it should) - The thread retains a reference to the web application class loader (this is the memory leak). The reference chain is: a) JDBC driver b) Thread c) Web application class loader d) Every class loaded by the web app Everything from c) onwards should be eligible for garbage collection but isn't because of the leak in the Thread that is retaining a reference to the web application class loader. This is what I would assume as correct behavior. If the thread is still attached ot the WebAppClassLoader, does that mean that the WebAppClassLoader cannot be garbage collected? The correct behaviour is that there is no reference to c) held by b) and therefore c) onwards would be eligible for GC (assuming the web app has been stopped). The problem here is that because b) has a reference to c), c) can't be GC'd and that is a memory leak. 2. Why does the JDBC Pool not unload the driver? That my cause the thread to stop after the last app has been stopped/undeployed. 1. Because the driver is in CATALINA_HOME/lib it is available to other applications. So first app, loads driver and keeps it in memory. Even if all apps are undeployed, driver remains cached and GCed when Tomcat is shut down? The JDBC driver will be loaded though the services API as soon as any code references the DriverManager. 2. JDBC drivers are never automatically unloaded. You have to do so explicitly (not doing so is an other source of memory leaks when the driver is packaged in WEB-INF/lib). OK, this won't be the case. Driver is always shared here. You need to go back to Oracle. Yes, I will. We're paying probably a lot of money for he company-wide license. I'd expect so. From my own experience with Oracle commercial support for things Java related, you'll have to go through the This is a bug. No it isn't. Yes it is... cycle several times before you manage to get the issue in front of someone with the necessary knowledge and skills to correctly identify this is a bug in Oracle's JDBC driver. I'd recommend escalating it through your account manager sooner rather than later. Feel free to point Oracle support to this thread and/or my presentation on memory leaks. If Oracle support require further help understanding this issue they are, of course, free to join this mailing list. Mark, I did receive an answer to the issue, citing your findings. See verbatim copy below: Hi Michael, I received the following
Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
On 21/05/2013 19:01, Michael-O wrote: Mark, I did receive an answer to the issue, citing your findings. See verbatim copy below: Hi Michael, I received the following update from our developer: ** The theoretical problem is that the thread is holding the app class loader so it remains reachable and the app is never unloaded. So if the user loads and unloads the app, the app classes remain in memory. Subsequent loading and unloading of the app will not get pinned in memory as the thread is already running so the subsequent loading and unloading will not cause additional class loaders to be pinned. It is a fixed size memory leak. It does not grow. The thread suggests setting the context class loader to be the parent of the default context class loader. This may work in Tomcat but it's pretty random. I am researching the problem to determine what if anything the driver should do to work across all containers. A Tomcat specific fix is not acceptable. Almost but not quite. The correct fix is to set the context class loader of the Thread to the class loader that loaded the Driver or, alternatively, the class loader that loaded the thread class. As Mark Thomas pointed out it is critical that the driver is loaded in the boot or system or container class loader, not the app class loader. If the driver is in the app class loader then when the app is unloaded the driver also should be unloaded. Unfortunately this doesn't work. The driver itself remains reachable so the app class loader is reachable so the app is never unloaded. At present there is no general way to solve this problem. The necessary hooks are added in JDK 8. I'd agree to this point. Most users put the driver in the container, not the app, so this is rarely a problem. I don't agree with this. I often see this with JDBC drivers which is why Tomcat has a pile of code specifically to unpick the mess this creates. ** I will certainly have to fill out a bug to have it investigated officially. That is good news. Seems like they understood the problem. But I do doubt that this is a fixed size moemory leak. I think the point they are trying to make is that it is only the first instance of the web application to be unloaded that will be pinned in memory. Subsequent reloads won't trigger a leak. That is correct. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
Am 2013-05-21 20:38, schrieb Mark Thomas: Seems like they understood the problem. But I do doubt that this is a fixed size moemory leak. I think the point they are trying to make is that it is only the first instance of the web application to be unloaded that will be pinned in memory. Subsequent reloads won't trigger a leak. That is correct. Does this mean that only one WebAppClassLoader stays in memory because the thread is launched once and attached to that class loader? Michael - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
On 21/05/2013 19:47, Michael-O wrote: Am 2013-05-21 20:38, schrieb Mark Thomas: Seems like they understood the problem. But I do doubt that this is a fixed size moemory leak. I think the point they are trying to make is that it is only the first instance of the web application to be unloaded that will be pinned in memory. Subsequent reloads won't trigger a leak. That is correct. Does this mean that only one WebAppClassLoader stays in memory because the thread is launched once and attached to that class loader? Exactly. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
Hi folks, there's now a follow-up on the issue [1]. Recap: JreMemoryLeakPreventionListener reported that my webapp has spawned OracleTimeoutPollingThread because I have set a QueryTimeout in the JDBC Pool. Mark Thomas noted that this is likely a bug in the driver. I have taken action and created a service request with Oracle. My personal analysis with VisualVM: The thread is spawned when the first query is run. The thread keeps running when the webapp is shut down or undeployed. Classes remain in memory. The JDBC Pool does a simple Class.forName to load the driver, it neither uses the DriverManager loading nor does it actively unload the driver. Oracle answer: They were able to reproduce the issue. The technical analysis says: Hi Michael, I confirmned internally that this message from Tomcat can be ignored as there is no risk of any real leak from OracleTimeoutPollingThread thread. This thread is related to the JDBC driver which may be used by many apps simultaneously. Unloading the app does not unload the driver and should not and does not stop the thread. It seems to be the design behaviour. So my questions would be: 1. Is that still not a false positive? 2. Why does the JDBC Pool not unload the driver? That my cause the thread to stop after the last app has been stopped/undeployed. Thanks, Michael [1] http://www.mail-archive.com/users@tomcat.apache.org/msg106186.html - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
On 17/05/2013 09:28, Michael-O wrote: Hi folks, there's now a follow-up on the issue [1]. Recap: JreMemoryLeakPreventionListener reported that my webapp has spawned OracleTimeoutPollingThread because I have set a QueryTimeout in the JDBC Pool. Mark Thomas noted that this is likely a bug in the driver. I have taken action and created a service request with Oracle. My personal analysis with VisualVM: The thread is spawned when the first query is run. The thread keeps running when the webapp is shut down or undeployed. Classes remain in memory. The JDBC Pool does a simple Class.forName to load the driver, it neither uses the DriverManager loading nor does it actively unload the driver. Oracle answer: They were able to reproduce the issue. The technical analysis says: Hi Michael, I confirmned internally that this message from Tomcat can be ignored as there is no risk of any real leak from OracleTimeoutPollingThread thread. This thread is related to the JDBC driver which may be used by many apps simultaneously. Unloading the app does not unload the driver and should not and does not stop the thread. It seems to be the design behaviour. So my questions would be: 1. Is that still not a false positive? No, that is not a false positive. The response from Oracle is wrong. There is nothing wrong with the driver creating the thread or the thread continuing after the web application has stopped. The problem is as follows: 1. When the JDBC driver creates the Thread, the Thread's context class loader (as returned by Thread.getContextClassLoader()) is set to the web application's class loader. 2. The correct behaviour at this point would be for the Driver to set the Thread's context class loader to the class loader that loaded the Driver class when the Thread is created. 3. The memory leak occurs as follows: - the web application is stopped - Tomcat clears all references to the web application and its classes - The web application should be eligible for garbage collection - The JDBC driver is still loaded (as it should be) - The JDBC driver retains a reference to the Thread (as it should) - The thread retains a reference to the web application class loader (this is the memory leak). The reference chain is: a) JDBC driver b) Thread c) Web application class loader d) Every class loaded by the web app Everything from c) onwards should be eligible for garbage collection but isn't because of the leak in the Thread that is retaining a reference to the web application class loader. 2. Why does the JDBC Pool not unload the driver? That my cause the thread to stop after the last app has been stopped/undeployed. 1. Because the driver is in CATALINA_HOME/lib it is available to other applications. 2. JDBC drivers are never automatically unloaded. You have to do so explicitly (not doing so is an other source of memory leaks when the driver is packaged in WEB-INF/lib). You need to go back to Oracle. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
Hi Mark, thanks again for the detailed answer, details inline. Gesendet: Freitag, 17. Mai 2013 um 11:36 Uhr Von: Mark Thomas ma...@apache.org An: Tomcat Users List users@tomcat.apache.org Betreff: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread On 17/05/2013 09:28, Michael-O wrote: Hi folks, there's now a follow-up on the issue [1]. Recap: JreMemoryLeakPreventionListener reported that my webapp has spawned OracleTimeoutPollingThread because I have set a QueryTimeout in the JDBC Pool. Mark Thomas noted that this is likely a bug in the driver. I have taken action and created a service request with Oracle. My personal analysis with VisualVM: The thread is spawned when the first query is run. The thread keeps running when the webapp is shut down or undeployed. Classes remain in memory. The JDBC Pool does a simple Class.forName to load the driver, it neither uses the DriverManager loading nor does it actively unload the driver. Oracle answer: They were able to reproduce the issue. The technical analysis says: Hi Michael, I confirmned internally that this message from Tomcat can be ignored as there is no risk of any real leak from OracleTimeoutPollingThread thread. This thread is related to the JDBC driver which may be used by many apps simultaneously. Unloading the app does not unload the driver and should not and does not stop the thread. It seems to be the design behaviour. So my questions would be: 1. Is that still not a false positive? No, that is not a false positive. The response from Oracle is wrong. There is nothing wrong with the driver creating the thread or the thread continuing after the web application has stopped. The problem is as follows: 1. When the JDBC driver creates the Thread, the Thread's context class loader (as returned by Thread.getContextClassLoader()) is set to the web application's class loader. 2. The correct behaviour at this point would be for the Driver to set the Thread's context class loader to the class loader that loaded the Driver class when the Thread is created. 3. The memory leak occurs as follows: - the web application is stopped - Tomcat clears all references to the web application and its classes - The web application should be eligible for garbage collection - The JDBC driver is still loaded (as it should be) - The JDBC driver retains a reference to the Thread (as it should) - The thread retains a reference to the web application class loader (this is the memory leak). The reference chain is: a) JDBC driver b) Thread c) Web application class loader d) Every class loaded by the web app Everything from c) onwards should be eligible for garbage collection but isn't because of the leak in the Thread that is retaining a reference to the web application class loader. This is what I would assume as correct behavior. If the thread is still attached ot the WebAppClassLoader, does that mean that the WebAppClassLoader cannot be garbage collected? 2. Why does the JDBC Pool not unload the driver? That my cause the thread to stop after the last app has been stopped/undeployed. 1. Because the driver is in CATALINA_HOME/lib it is available to other applications. So first app, loads driver and keeps it in memory. Even if all apps are undeployed, driver remains cached and GCed when Tomcat is shut down? 2. JDBC drivers are never automatically unloaded. You have to do so explicitly (not doing so is an other source of memory leaks when the driver is packaged in WEB-INF/lib). OK, this won't be the case. Driver is always shared here. You need to go back to Oracle. Yes, I will. We're paying probably a lot of money for he company-wide license. Thanks again, Michael - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
On 17/05/2013 12:31, Michael-O wrote: Hi Mark, thanks again for the detailed answer, details inline. Gesendet: Freitag, 17. Mai 2013 um 11:36 Uhr Von: Mark Thomas ma...@apache.org An: Tomcat Users List users@tomcat.apache.org Betreff: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread On 17/05/2013 09:28, Michael-O wrote: Hi folks, there's now a follow-up on the issue [1]. Recap: JreMemoryLeakPreventionListener reported that my webapp has spawned OracleTimeoutPollingThread because I have set a QueryTimeout in the JDBC Pool. Mark Thomas noted that this is likely a bug in the driver. I have taken action and created a service request with Oracle. My personal analysis with VisualVM: The thread is spawned when the first query is run. The thread keeps running when the webapp is shut down or undeployed. Classes remain in memory. The JDBC Pool does a simple Class.forName to load the driver, it neither uses the DriverManager loading nor does it actively unload the driver. Oracle answer: They were able to reproduce the issue. The technical analysis says: Hi Michael, I confirmned internally that this message from Tomcat can be ignored as there is no risk of any real leak from OracleTimeoutPollingThread thread. This thread is related to the JDBC driver which may be used by many apps simultaneously. Unloading the app does not unload the driver and should not and does not stop the thread. It seems to be the design behaviour. So my questions would be: 1. Is that still not a false positive? No, that is not a false positive. The response from Oracle is wrong. There is nothing wrong with the driver creating the thread or the thread continuing after the web application has stopped. The problem is as follows: 1. When the JDBC driver creates the Thread, the Thread's context class loader (as returned by Thread.getContextClassLoader()) is set to the web application's class loader. 2. The correct behaviour at this point would be for the Driver to set the Thread's context class loader to the class loader that loaded the Driver class when the Thread is created. 3. The memory leak occurs as follows: - the web application is stopped - Tomcat clears all references to the web application and its classes - The web application should be eligible for garbage collection - The JDBC driver is still loaded (as it should be) - The JDBC driver retains a reference to the Thread (as it should) - The thread retains a reference to the web application class loader (this is the memory leak). The reference chain is: a) JDBC driver b) Thread c) Web application class loader d) Every class loaded by the web app Everything from c) onwards should be eligible for garbage collection but isn't because of the leak in the Thread that is retaining a reference to the web application class loader. This is what I would assume as correct behavior. If the thread is still attached ot the WebAppClassLoader, does that mean that the WebAppClassLoader cannot be garbage collected? The correct behaviour is that there is no reference to c) held by b) and therefore c) onwards would be eligible for GC (assuming the web app has been stopped). The problem here is that because b) has a reference to c), c) can't be GC'd and that is a memory leak. 2. Why does the JDBC Pool not unload the driver? That my cause the thread to stop after the last app has been stopped/undeployed. 1. Because the driver is in CATALINA_HOME/lib it is available to other applications. So first app, loads driver and keeps it in memory. Even if all apps are undeployed, driver remains cached and GCed when Tomcat is shut down? The JDBC driver will be loaded though the services API as soon as any code references the DriverManager. 2. JDBC drivers are never automatically unloaded. You have to do so explicitly (not doing so is an other source of memory leaks when the driver is packaged in WEB-INF/lib). OK, this won't be the case. Driver is always shared here. You need to go back to Oracle. Yes, I will. We're paying probably a lot of money for he company-wide license. I'd expect so. From my own experience with Oracle commercial support for things Java related, you'll have to go through the This is a bug. No it isn't. Yes it is... cycle several times before you manage to get the issue in front of someone with the necessary knowledge and skills to correctly identify this is a bug in Oracle's JDBC driver. I'd recommend escalating it through your account manager sooner rather than later. Feel free to point Oracle support to this thread and/or my presentation on memory leaks. If Oracle support require further help understanding this issue they are, of course, free to join this mailing list. Mark - To unsubscribe, e-mail: users-unsubscr
Aw: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
Gesendet: Freitag, 17. Mai 2013 um 14:26 Uhr Von: Mark Thomas ma...@apache.org On 17/05/2013 12:31, Michael-O wrote: Hi Mark, thanks again for the detailed answer, details inline. Gesendet: Freitag, 17. Mai 2013 um 11:36 Uhr Von: Mark Thomas ma...@apache.org An: Tomcat Users List users@tomcat.apache.org Betreff: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread On 17/05/2013 09:28, Michael-O wrote: Hi folks, there's now a follow-up on the issue [1]. Recap: JreMemoryLeakPreventionListener reported that my webapp has spawned OracleTimeoutPollingThread because I have set a QueryTimeout in the JDBC Pool. Mark Thomas noted that this is likely a bug in the driver. I have taken action and created a service request with Oracle. My personal analysis with VisualVM: The thread is spawned when the first query is run. The thread keeps running when the webapp is shut down or undeployed. Classes remain in memory. The JDBC Pool does a simple Class.forName to load the driver, it neither uses the DriverManager loading nor does it actively unload the driver. Oracle answer: They were able to reproduce the issue. The technical analysis says: Hi Michael, I confirmned internally that this message from Tomcat can be ignored as there is no risk of any real leak from OracleTimeoutPollingThread thread. This thread is related to the JDBC driver which may be used by many apps simultaneously. Unloading the app does not unload the driver and should not and does not stop the thread. It seems to be the design behaviour. So my questions would be: 1. Is that still not a false positive? No, that is not a false positive. The response from Oracle is wrong. There is nothing wrong with the driver creating the thread or the thread continuing after the web application has stopped. The problem is as follows: 1. When the JDBC driver creates the Thread, the Thread's context class loader (as returned by Thread.getContextClassLoader()) is set to the web application's class loader. 2. The correct behaviour at this point would be for the Driver to set the Thread's context class loader to the class loader that loaded the Driver class when the Thread is created. 3. The memory leak occurs as follows: - the web application is stopped - Tomcat clears all references to the web application and its classes - The web application should be eligible for garbage collection - The JDBC driver is still loaded (as it should be) - The JDBC driver retains a reference to the Thread (as it should) - The thread retains a reference to the web application class loader (this is the memory leak). The reference chain is: a) JDBC driver b) Thread c) Web application class loader d) Every class loaded by the web app Everything from c) onwards should be eligible for garbage collection but isn't because of the leak in the Thread that is retaining a reference to the web application class loader. This is what I would assume as correct behavior. If the thread is still attached ot the WebAppClassLoader, does that mean that the WebAppClassLoader cannot be garbage collected? The correct behaviour is that there is no reference to c) held by b) and therefore c) onwards would be eligible for GC (assuming the web app has been stopped). The problem here is that because b) has a reference to c), c) can't be GC'd and that is a memory leak. Exactly, WebAppClassLoader keeps in a stale status. 2. Why does the JDBC Pool not unload the driver? That my cause the thread to stop after the last app has been stopped/undeployed. 1. Because the driver is in CATALINA_HOME/lib it is available to other applications. So first app, loads driver and keeps it in memory. Even if all apps are undeployed, driver remains cached and GCed when Tomcat is shut down? The JDBC driver will be loaded though the services API as soon as any code references the DriverManager. 2. JDBC drivers are never automatically unloaded. You have to do so explicitly (not doing so is an other source of memory leaks when the driver is packaged in WEB-INF/lib). OK, this won't be the case. Driver is always shared here. You need to go back to Oracle. Yes, I will. We're paying probably a lot of money for he company-wide license. I'd expect so. From my own experience with Oracle commercial support for things Java related, you'll have to go through the This is a bug. No it isn't. Yes it is... cycle several times before you manage to get the issue in front of someone with the necessary knowledge and skills to correctly identify this is a bug in Oracle's JDBC driver. I'd recommend escalating it through your account manager sooner rather than later. Feel free to point Oracle support to this thread and/or my
RE: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
-Original Message- From: Mark Thomas [mailto:ma...@apache.org] Sent: Friday, May 17, 2013 7:26 AM To: Tomcat Users List Subject: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread On 17/05/2013 12:31, Michael-O wrote: Hi Mark, thanks again for the detailed answer, details inline. Gesendet: Freitag, 17. Mai 2013 um 11:36 Uhr Von: Mark Thomas ma...@apache.org An: Tomcat Users List users@tomcat.apache.org Betreff: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread On 17/05/2013 09:28, Michael-O wrote: Hi folks, there's now a follow-up on the issue [1]. Recap: JreMemoryLeakPreventionListener reported that my webapp has spawned OracleTimeoutPollingThread because I have set a QueryTimeout in the JDBC Pool. Mark Thomas noted that this is likely a bug in the driver. I have taken action and created a service request with Oracle. My personal analysis with VisualVM: The thread is spawned when the first query is run. The thread keeps running when the webapp is shut down or undeployed. Classes remain in memory. The JDBC Pool does a simple Class.forName to load the driver, it neither uses the DriverManager loading nor does it actively unload the driver. Oracle answer: They were able to reproduce the issue. The technical analysis says: Hi Michael, I confirmned internally that this message from Tomcat can be ignored as there is no risk of any real leak from OracleTimeoutPollingThread thread. This thread is related to the JDBC driver which may be used by many apps simultaneously. Unloading the app does not unload the driver and should not and does not stop the thread. It seems to be the design behaviour. So my questions would be: 1. Is that still not a false positive? No, that is not a false positive. The response from Oracle is wrong. There is nothing wrong with the driver creating the thread or the thread continuing after the web application has stopped. The problem is as follows: 1. When the JDBC driver creates the Thread, the Thread's context class loader (as returned by Thread.getContextClassLoader()) is set to the web application's class loader. 2. The correct behaviour at this point would be for the Driver to set the Thread's context class loader to the class loader that loaded the Driver class when the Thread is created. 3. The memory leak occurs as follows: - the web application is stopped - Tomcat clears all references to the web application and its classes - The web application should be eligible for garbage collection - The JDBC driver is still loaded (as it should be) - The JDBC driver retains a reference to the Thread (as it should) - The thread retains a reference to the web application class loader (this is the memory leak). The reference chain is: a) JDBC driver b) Thread c) Web application class loader d) Every class loaded by the web app Everything from c) onwards should be eligible for garbage collection but isn't because of the leak in the Thread that is retaining a reference to the web application class loader. This is what I would assume as correct behavior. If the thread is still attached ot the WebAppClassLoader, does that mean that the WebAppClassLoader cannot be garbage collected? The correct behaviour is that there is no reference to c) held by b) and therefore c) onwards would be eligible for GC (assuming the web app has been stopped). The problem here is that because b) has a reference to c), c) can't be GC'd and that is a memory leak. 2. Why does the JDBC Pool not unload the driver? That my cause the thread to stop after the last app has been stopped/undeployed. 1. Because the driver is in CATALINA_HOME/lib it is available to other applications. So first app, loads driver and keeps it in memory. Even if all apps are undeployed, driver remains cached and GCed when Tomcat is shut down? The JDBC driver will be loaded though the services API as soon as any code references the DriverManager. 2. JDBC drivers are never automatically unloaded. You have to do so explicitly (not doing so is an other source of memory leaks when the driver is packaged in WEB-INF/lib). OK, this won't be the case. Driver is always shared here. You need to go back to Oracle. Yes, I will. We're paying probably a lot of money for he company-wide license. I'd expect so. From my own experience with Oracle commercial support for things Java related, you'll have to go through the This is a bug. No it isn't. Yes it is... cycle several times before you manage to get the issue in front of someone with the necessary knowledge and skills to correctly identify this is a bug in Oracle's JDBC driver. I'd recommend escalating it through your
Aw: RE: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
-Original Message- From: Mark Thomas [mailto:ma...@apache.org] Sent: Friday, May 17, 2013 7:26 AM To: Tomcat Users List Subject: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread On 17/05/2013 12:31, Michael-O wrote: Hi Mark, thanks again for the detailed answer, details inline. Gesendet: Freitag, 17. Mai 2013 um 11:36 Uhr Von: Mark Thomas ma...@apache.org An: Tomcat Users List users@tomcat.apache.org Betreff: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread On 17/05/2013 09:28, Michael-O wrote: Hi folks, there's now a follow-up on the issue [1]. Recap: JreMemoryLeakPreventionListener reported that my webapp has spawned OracleTimeoutPollingThread because I have set a QueryTimeout in the JDBC Pool. Mark Thomas noted that this is likely a bug in the driver. I have taken action and created a service request with Oracle. My personal analysis with VisualVM: The thread is spawned when the first query is run. The thread keeps running when the webapp is shut down or undeployed. Classes remain in memory. The JDBC Pool does a simple Class.forName to load the driver, it neither uses the DriverManager loading nor does it actively unload the driver. Oracle answer: They were able to reproduce the issue. The technical analysis says: Hi Michael, I confirmned internally that this message from Tomcat can be ignored as there is no risk of any real leak from OracleTimeoutPollingThread thread. This thread is related to the JDBC driver which may be used by many apps simultaneously. Unloading the app does not unload the driver and should not and does not stop the thread. It seems to be the design behaviour. So my questions would be: 1. Is that still not a false positive? No, that is not a false positive. The response from Oracle is wrong. There is nothing wrong with the driver creating the thread or the thread continuing after the web application has stopped. The problem is as follows: 1. When the JDBC driver creates the Thread, the Thread's context class loader (as returned by Thread.getContextClassLoader()) is set to the web application's class loader. 2. The correct behaviour at this point would be for the Driver to set the Thread's context class loader to the class loader that loaded the Driver class when the Thread is created. 3. The memory leak occurs as follows: - the web application is stopped - Tomcat clears all references to the web application and its classes - The web application should be eligible for garbage collection - The JDBC driver is still loaded (as it should be) - The JDBC driver retains a reference to the Thread (as it should) - The thread retains a reference to the web application class loader (this is the memory leak). The reference chain is: a) JDBC driver b) Thread c) Web application class loader d) Every class loaded by the web app Everything from c) onwards should be eligible for garbage collection but isn't because of the leak in the Thread that is retaining a reference to the web application class loader. This is what I would assume as correct behavior. If the thread is still attached ot the WebAppClassLoader, does that mean that the WebAppClassLoader cannot be garbage collected? The correct behaviour is that there is no reference to c) held by b) and therefore c) onwards would be eligible for GC (assuming the web app has been stopped). The problem here is that because b) has a reference to c), c) can't be GC'd and that is a memory leak. 2. Why does the JDBC Pool not unload the driver? That my cause the thread to stop after the last app has been stopped/undeployed. 1. Because the driver is in CATALINA_HOME/lib it is available to other applications. So first app, loads driver and keeps it in memory. Even if all apps are undeployed, driver remains cached and GCed when Tomcat is shut down? The JDBC driver will be loaded though the services API as soon as any code references the DriverManager. 2. JDBC drivers are never automatically unloaded. You have to do so explicitly (not doing so is an other source of memory leaks when the driver is packaged in WEB-INF/lib). OK, this won't be the case. Driver is always shared here. You need to go back to Oracle. Yes, I will. We're paying probably a lot of money for he company-wide license. I'd expect so. From my own experience with Oracle commercial support for things Java related, you'll have to go through the This is a bug. No it isn't. Yes it is... cycle several times before you manage to get the issue in front of someone
Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
On 17/05/2013 15:34, Jeffrey Janner wrote: Michael and Mark - I happened to be reviewing how Oracle handles QueryTimeout yesterday on an unrelated issue and came across a passage in the JDBC Developers Guide (11g) that covered this Monitoring Thread (page E-3). My reading of it was that Oracle creates a single monitoring thread per JVM, so if there are multiple apps utilizing the Oracle driver, you certainly don't want one app shutting down and taking the monitor thread with it. I'm sure this one of the reasons that Mark says that the monitor thread should use the parent class loader. Oracle should probably consider this. My question though is how is all this affected when the jdbc library is loaded on a per-app basis, i.e., it's not shared but loaded from each app's WEB-INF/lib folder? Should the app then unload the driver when it shuts down to avoid memory leaks? Or is there a special process that needs to be followed? Jeff If the driver is in WEB-INF/lib things get really messy, very quickly. You can make it work if only that application is using the driver (you have to make sure you manually deregister the driver in a ServletContextListener). If multiple apps try and use it, or if multiple apps ship the same Driver, bad things happen. There are multiple problems: 1. Using a driver automatically adds it to the DriverManager. DriverManager is a JVM singleton. It maintains a list of loaded drivers but does not key this list by class loader so there can only ever be one instance of a Driver loaded even if different web applications ship different versions of the same Driver. 2. If web app A loads the Driver and web app B tries to use it the chances of a CNFE or similar are fairly high. You might get away with it. You probably won't. 3. If webapp A loads the driver, webapp B manages to use it and then webapp A is undeployed, assuming webapp A does the right thing and deregisaters it, webapp A will pull the driver out from underneath webapp B. Generally, putting it in CATALINA_HOME/lib is better. That also means Tomcat can provide database connection pooling. DriverManager is just one of the many JRE classes that make no allowances for running in the class loader environments you get in J2EE containers. LogManager is another. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
Hi folks, I recently enabled a QueryTimeoutInterceptor with queryTimeout of 60 seconds in a JDBC Pool data source (7.0.37). When the app was shut down, Tomcat said: The web application [/...] appears to have started a thread named [OracleTimeoutPollingThread] but has failed to stop it... We are using Oracle 11.2g with 11.2.0.3 JDBC drivers. I have figured out that this thread is spawned by the driver itself. According to this Stackoverflow answer [1] this is a long-living thread, same says the JDBC FAQ [2] of Oracle. The thread seems to work like Pool's PoolCleaner thread. A few month a ago I reported the same issue with the PoolCleaner thread and Filip fixed the class loader orders. Can this be a false-positive by the memory leak detector since this thread lives only once in the entire VM? Thanks, Michael [1] http://stackoverflow.com/a/3387312/696632 [2] http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-faq-090281.html#02_14 - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
On 07/05/2013 09:16, Michael-O wrote: Hi folks, I recently enabled a QueryTimeoutInterceptor with queryTimeout of 60 seconds in a JDBC Pool data source (7.0.37). When the app was shut down, Tomcat said: The web application [/...] appears to have started a thread named [OracleTimeoutPollingThread] but has failed to stop it... We are using Oracle 11.2g with 11.2.0.3 JDBC drivers. I have figured out that this thread is spawned by the driver itself. According to this Stackoverflow answer [1] this is a long-living thread, same says the JDBC FAQ [2] of Oracle. The thread seems to work like Pool's PoolCleaner thread. A few month a ago I reported the same issue with the PoolCleaner thread and Filip fixed the class loader orders. Can this be a false-positive by the memory leak detector since this thread lives only once in the entire VM? No. It is a memory leak and either or bug in your application or in the JDBC driver. Where is the Oracle JDBC driver? CATALINA_[BASE|HOME]/lib or WEB-INF/lib Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Aw: Re: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
Von: Mark Thomas ma...@apache.org On 07/05/2013 09:16, Michael-O wrote: Hi folks, I recently enabled a QueryTimeoutInterceptor with queryTimeout of 60 seconds in a JDBC Pool data source (7.0.37). When the app was shut down, Tomcat said: The web application [/...] appears to have started a thread named [OracleTimeoutPollingThread] but has failed to stop it... We are using Oracle 11.2g with 11.2.0.3 JDBC drivers. I have figured out that this thread is spawned by the driver itself. According to this Stackoverflow answer [1] this is a long-living thread, same says the JDBC FAQ [2] of Oracle. The thread seems to work like Pool's PoolCleaner thread. A few month a ago I reported the same issue with the PoolCleaner thread and Filip fixed the class loader orders. Can this be a false-positive by the memory leak detector since this thread lives only once in the entire VM? No. It is a memory leak and either or bug in your application or in the JDBC driver. Where is the Oracle JDBC driver? CATALINA_[BASE|HOME]/lib or WEB-INF/lib The driver is in the $CATALINA_HOME/lib only where $CATALINA_BASE != $CATALINA_HOME. This was done for a single webapp for testing purposes. Does this make a difference? How do you know that this is not a false-positive? If you really know for sure, I can open a service request with Oracle Support. Michael - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Aw: Re: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
On 07/05/2013 10:25, Michael-O wrote: Von: Mark Thomas ma...@apache.org On 07/05/2013 09:16, Michael-O wrote: Hi folks, I recently enabled a QueryTimeoutInterceptor with queryTimeout of 60 seconds in a JDBC Pool data source (7.0.37). When the app was shut down, Tomcat said: The web application [/...] appears to have started a thread named [OracleTimeoutPollingThread] but has failed to stop it... We are using Oracle 11.2g with 11.2.0.3 JDBC drivers. I have figured out that this thread is spawned by the driver itself. According to this Stackoverflow answer [1] this is a long-living thread, same says the JDBC FAQ [2] of Oracle. The thread seems to work like Pool's PoolCleaner thread. A few month a ago I reported the same issue with the PoolCleaner thread and Filip fixed the class loader orders. Can this be a false-positive by the memory leak detector since this thread lives only once in the entire VM? No. It is a memory leak and either or bug in your application or in the JDBC driver. Where is the Oracle JDBC driver? CATALINA_[BASE|HOME]/lib or WEB-INF/lib The driver is in the $CATALINA_HOME/lib only where $CATALINA_BASE != $CATALINA_HOME. This was done for a single webapp for testing purposes. Does this make a difference? The important thing is that it isn't in WEB-INF/lib. How do you know that this is not a false-positive? Experience, a lot of research into memory leaks and I wrote Tomcat's memory leak detection code. If you really know for sure, I can open a service request with Oracle Support. Good luck with that. The problem is that when the driver creates the thread it does so when the current class loader is the web application class loader. That means that the Thread will be created with a context class loader set to the web application class loader. That will trigger a memory leak when the web application is stopped because a reference is retained to the web application's class loader. What the driver should do is, after it creates the thread, set the thread's context class loader to the class loader that loaded the driver. What you are seeing is a variation of the leak described on page 15 of [1]. Mark [1] http://people.apache.org/~markt/presentations/2010-11-04-Memory-Leaks-60mins.pdf - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Re: Re: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
Von: Mark Thomas ma...@apache.org On 07/05/2013 10:25, Michael-O wrote: Von: Mark Thomas ma...@apache.org On 07/05/2013 09:16, Michael-O wrote: Hi folks, I recently enabled a QueryTimeoutInterceptor with queryTimeout of 60 seconds in a JDBC Pool data source (7.0.37). When the app was shut down, Tomcat said: The web application [/...] appears to have started a thread named [OracleTimeoutPollingThread] but has failed to stop it... We are using Oracle 11.2g with 11.2.0.3 JDBC drivers. I have figured out that this thread is spawned by the driver itself. According to this Stackoverflow answer [1] this is a long-living thread, same says the JDBC FAQ [2] of Oracle. The thread seems to work like Pool's PoolCleaner thread. A few month a ago I reported the same issue with the PoolCleaner thread and Filip fixed the class loader orders. Can this be a false-positive by the memory leak detector since this thread lives only once in the entire VM? No. It is a memory leak and either or bug in your application or in the JDBC driver. Where is the Oracle JDBC driver? CATALINA_[BASE|HOME]/lib or WEB-INF/lib The driver is in the $CATALINA_HOME/lib only where $CATALINA_BASE != $CATALINA_HOME. This was done for a single webapp for testing purposes. Does this make a difference? The important thing is that it isn't in WEB-INF/lib. How do you know that this is not a false-positive? Experience, a lot of research into memory leaks and I wrote Tomcat's memory leak detection code. If you really know for sure, I can open a service request with Oracle Support. Good luck with that. The problem is that when the driver creates the thread it does so when the current class loader is the web application class loader. That means that the Thread will be created with a context class loader set to the web application class loader. That will trigger a memory leak when the web application is stopped because a reference is retained to the web application's class loader. What the driver should do is, after it creates the thread, set the thread's context class loader to the class loader that loaded the driver. What you are seeing is a variation of the leak described on page 15 of [1]. After reading the slides and your explanation this makes sense. It's the same issue as Pool cleaner thread should be created using the classloader that loaded the pool, not the context loader (fhanik) fixed in 7.0.27. I will file SR and let you know. Michael - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Aw: Re: Re: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
Von: Mark Thomas ma...@apache.org On 07/05/2013 10:25, Michael-O wrote: Von: Mark Thomas ma...@apache.org On 07/05/2013 09:16, Michael-O wrote: Hi folks, I recently enabled a QueryTimeoutInterceptor with queryTimeout of 60 seconds in a JDBC Pool data source (7.0.37). When the app was shut down, Tomcat said: The web application [/...] appears to have started a thread named [OracleTimeoutPollingThread] but has failed to stop it... We are using Oracle 11.2g with 11.2.0.3 JDBC drivers. I have figured out that this thread is spawned by the driver itself. According to this Stackoverflow answer [1] this is a long-living thread, same says the JDBC FAQ [2] of Oracle. The thread seems to work like Pool's PoolCleaner thread. A few month a ago I reported the same issue with the PoolCleaner thread and Filip fixed the class loader orders. Can this be a false-positive by the memory leak detector since this thread lives only once in the entire VM? No. It is a memory leak and either or bug in your application or in the JDBC driver. Where is the Oracle JDBC driver? CATALINA_[BASE|HOME]/lib or WEB-INF/lib The driver is in the $CATALINA_HOME/lib only where $CATALINA_BASE != $CATALINA_HOME. This was done for a single webapp for testing purposes. Does this make a difference? The important thing is that it isn't in WEB-INF/lib. How do you know that this is not a false-positive? Experience, a lot of research into memory leaks and I wrote Tomcat's memory leak detection code. If you really know for sure, I can open a service request with Oracle Support. Good luck with that. SR created. I will keep you updated. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Michael, On 5/7/13 6:43 AM, Michael-O wrote: Von: Mark Thomas ma...@apache.org On 07/05/2013 10:25, Michael-O wrote: Von: Mark Thomas ma...@apache.org On 07/05/2013 09:16, Michael-O wrote: Hi folks, I recently enabled a QueryTimeoutInterceptor with queryTimeout of 60 seconds in a JDBC Pool data source (7.0.37). When the app was shut down, Tomcat said: The web application [/...] appears to have started a thread named [OracleTimeoutPollingThread] but has failed to stop it... We are using Oracle 11.2g with 11.2.0.3 JDBC drivers. I have figured out that this thread is spawned by the driver itself. According to this Stackoverflow answer [1] this is a long-living thread, same says the JDBC FAQ [2] of Oracle. The thread seems to work like Pool's PoolCleaner thread. A few month a ago I reported the same issue with the PoolCleaner thread and Filip fixed the class loader orders. Can this be a false-positive by the memory leak detector since this thread lives only once in the entire VM? No. It is a memory leak and either or bug in your application or in the JDBC driver. Where is the Oracle JDBC driver? CATALINA_[BASE|HOME]/lib or WEB-INF/lib The driver is in the $CATALINA_HOME/lib only where $CATALINA_BASE != $CATALINA_HOME. This was done for a single webapp for testing purposes. Does this make a difference? The important thing is that it isn't in WEB-INF/lib. How do you know that this is not a false-positive? Experience, a lot of research into memory leaks and I wrote Tomcat's memory leak detection code. If you really know for sure, I can open a service request with Oracle Support. Good luck with that. The problem is that when the driver creates the thread it does so when the current class loader is the web application class loader. That means that the Thread will be created with a context class loader set to the web application class loader. That will trigger a memory leak when the web application is stopped because a reference is retained to the web application's class loader. What the driver should do is, after it creates the thread, set the thread's context class loader to the class loader that loaded the driver. What you are seeing is a variation of the leak described on page 15 of [1]. After reading the slides and your explanation this makes sense. It's the same issue as Pool cleaner thread should be created using the classloader that loaded the pool, not the context loader (fhanik) fixed in 7.0.27. I will file SR and let you know. Note that you might be able to write your own code to mitigate this problem, depending on exactly when that thread is created. If the timeout thread isn't created until you actually try to issue a query with a timeout, try something like this in a ServletContextListener's contextInitialized method: // NOTE: No resource management is being done in this example // Get the current ClassLoader -- should be WebappClassLoader ClassLoader cl = Thread.currentThread().getContextClassLoader(); // WebappClassLoader.getParent should be common loader Thread.currentThread().setContextClassLoader(cl.getParent()); try { Connection conn = ...; // However you get a connection Statement s = conn.createStatement(); s.setQueryTimeout(5000); // Doesn't really matter what TO is s.execute(SELECT 1 FROM dual, ); } finally { // Pop back to the original ClassLoader Thread.currentThread().setContextClassLoader(cl); } This is the exact technique that Tomcat's JreMemoryLeakPreventionListener uses to prevent ClassLoader leaks. A couple of notes: 1. This won't work under a SecurityManager. If you need to operate under a SecurityManager, have a look at the JreMemoryLeakPreventionListener code and adapt it to the above. 2. If the Oracle driver launches the thread when the DataSource is created, it might happen too early for a ServletContextListener to intervene. In that case, simply modify the JreMEmoryLeakPreventionListener code directly. Patches are always welcome. 3. If Oracle fixes this bug, Tomcat should not prevent against it. If Oracle refuses to acknowledge/fix/etc. this bug, then it may make sense to include such code in JreMemoryLeakPreventionListener, but it should probably be done in such a way that a) it's not enabled by default and b) the query used for triggering the Thread to be created is user-selectable with maybe a reasonable default (like SELECT 1 FROM dual, as that tends to be valid in most RDBMSs). - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJRiRulAAoJEBzwKT+lPKRYQCUP/jOhPNIje1C114EOvDkGVb7T jyx+LiULlos6ZE3eMlu889tC7rjRZVRp39TEs8qMW42JT6etLAeY2bwWtqenbM7m
Re: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
Hello Christopher, Am 2013-05-07 17:20, schrieb Christopher Schultz: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Michael, On 5/7/13 6:43 AM, Michael-O wrote: Von: Mark Thomas ma...@apache.org On 07/05/2013 10:25, Michael-O wrote: Von: Mark Thomas ma...@apache.org On 07/05/2013 09:16, Michael-O wrote: Hi folks, I recently enabled a QueryTimeoutInterceptor with queryTimeout of 60 seconds in a JDBC Pool data source (7.0.37). When the app was shut down, Tomcat said: The web application [/...] appears to have started a thread named [OracleTimeoutPollingThread] but has failed to stop it... We are using Oracle 11.2g with 11.2.0.3 JDBC drivers. I have figured out that this thread is spawned by the driver itself. According to this Stackoverflow answer [1] this is a long-living thread, same says the JDBC FAQ [2] of Oracle. The thread seems to work like Pool's PoolCleaner thread. A few month a ago I reported the same issue with the PoolCleaner thread and Filip fixed the class loader orders. Can this be a false-positive by the memory leak detector since this thread lives only once in the entire VM? No. It is a memory leak and either or bug in your application or in the JDBC driver. Where is the Oracle JDBC driver? CATALINA_[BASE|HOME]/lib or WEB-INF/lib The driver is in the $CATALINA_HOME/lib only where $CATALINA_BASE != $CATALINA_HOME. This was done for a single webapp for testing purposes. Does this make a difference? The important thing is that it isn't in WEB-INF/lib. How do you know that this is not a false-positive? Experience, a lot of research into memory leaks and I wrote Tomcat's memory leak detection code. If you really know for sure, I can open a service request with Oracle Support. Good luck with that. The problem is that when the driver creates the thread it does so when the current class loader is the web application class loader. That means that the Thread will be created with a context class loader set to the web application class loader. That will trigger a memory leak when the web application is stopped because a reference is retained to the web application's class loader. What the driver should do is, after it creates the thread, set the thread's context class loader to the class loader that loaded the driver. What you are seeing is a variation of the leak described on page 15 of [1]. After reading the slides and your explanation this makes sense. It's the same issue as Pool cleaner thread should be created using the classloader that loaded the pool, not the context loader (fhanik) fixed in 7.0.27. I will file SR and let you know. Note that you might be able to write your own code to mitigate this problem, depending on exactly when that thread is created. If the timeout thread isn't created until you actually try to issue a query with a timeout, try something like this in a ServletContextListener's contextInitialized method: // NOTE: No resource management is being done in this example // Get the current ClassLoader -- should be WebappClassLoader ClassLoader cl = Thread.currentThread().getContextClassLoader(); // WebappClassLoader.getParent should be common loader Thread.currentThread().setContextClassLoader(cl.getParent()); try { Connection conn = ...; // However you get a connection Statement s = conn.createStatement(); s.setQueryTimeout(5000); // Doesn't really matter what TO is s.execute(SELECT 1 FROM dual, ); } finally { // Pop back to the original ClassLoader Thread.currentThread().setContextClassLoader(cl); } This is the exact technique that Tomcat's JreMemoryLeakPreventionListener uses to prevent ClassLoader leaks. Yes, this looks like the way JDBC Pool does with the Pool Cleaner. I would go for this as a last resort. A couple of notes: 1. This won't work under a SecurityManager. If you need to operate under a SecurityManager, have a look at the JreMemoryLeakPreventionListener code and adapt it to the above. 2. If the Oracle driver launches the thread when the DataSource is created, it might happen too early for a ServletContextListener to intervene. In that case, simply modify the JreMEmoryLeakPreventionListener code directly. Patches are always welcome. Not, it is not. I have attached VisualVM to Tomcat VM and have seen that the thread is forked when the first query is executed. 3. If Oracle fixes this bug, Tomcat should not prevent against it. If Oracle refuses to acknowledge/fix/etc. this bug, then it may make sense to include such code in JreMemoryLeakPreventionListener, but it should probably be done in such a way that a) it's not enabled by default and b) the query used for triggering the Thread to be created is user-selectable with maybe a reasonable default (like SELECT 1 FROM dual, as that tends to be valid in most RDBMSs). I am already in contact with an Oracle engineer who has received a demo WAR file to
Re: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Michael, On 5/7/13 4:33 PM, Michael-O wrote: Am 2013-05-07 17:20, schrieb Christopher Schultz: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Michael, On 5/7/13 6:43 AM, Michael-O wrote: Von: Mark Thomas ma...@apache.org On 07/05/2013 10:25, Michael-O wrote: Von: Mark Thomas ma...@apache.org On 07/05/2013 09:16, Michael-O wrote: Hi folks, I recently enabled a QueryTimeoutInterceptor with queryTimeout of 60 seconds in a JDBC Pool data source (7.0.37). When the app was shut down, Tomcat said: The web application [/...] appears to have started a thread named [OracleTimeoutPollingThread] but has failed to stop it... We are using Oracle 11.2g with 11.2.0.3 JDBC drivers. I have figured out that this thread is spawned by the driver itself. According to this Stackoverflow answer [1] this is a long-living thread, same says the JDBC FAQ [2] of Oracle. The thread seems to work like Pool's PoolCleaner thread. A few month a ago I reported the same issue with the PoolCleaner thread and Filip fixed the class loader orders. Can this be a false-positive by the memory leak detector since this thread lives only once in the entire VM? No. It is a memory leak and either or bug in your application or in the JDBC driver. Where is the Oracle JDBC driver? CATALINA_[BASE|HOME]/lib or WEB-INF/lib The driver is in the $CATALINA_HOME/lib only where $CATALINA_BASE != $CATALINA_HOME. This was done for a single webapp for testing purposes. Does this make a difference? The important thing is that it isn't in WEB-INF/lib. How do you know that this is not a false-positive? Experience, a lot of research into memory leaks and I wrote Tomcat's memory leak detection code. If you really know for sure, I can open a service request with Oracle Support. Good luck with that. The problem is that when the driver creates the thread it does so when the current class loader is the web application class loader. That means that the Thread will be created with a context class loader set to the web application class loader. That will trigger a memory leak when the web application is stopped because a reference is retained to the web application's class loader. What the driver should do is, after it creates the thread, set the thread's context class loader to the class loader that loaded the driver. What you are seeing is a variation of the leak described on page 15 of [1]. After reading the slides and your explanation this makes sense. It's the same issue as Pool cleaner thread should be created using the classloader that loaded the pool, not the context loader (fhanik) fixed in 7.0.27. I will file SR and let you know. Note that you might be able to write your own code to mitigate this problem, depending on exactly when that thread is created. If the timeout thread isn't created until you actually try to issue a query with a timeout, try something like this in a ServletContextListener's contextInitialized method: // NOTE: No resource management is being done in this example // Get the current ClassLoader -- should be WebappClassLoader ClassLoader cl = Thread.currentThread().getContextClassLoader(); // WebappClassLoader.getParent should be common loader Thread.currentThread().setContextClassLoader(cl.getParent()); try { Connection conn = ...; // However you get a connection Statement s = conn.createStatement(); s.setQueryTimeout(5000); // Doesn't really matter what TO is s.execute(SELECT 1 FROM dual, ); } finally { // Pop back to the original ClassLoader Thread.currentThread().setContextClassLoader(cl); } This is the exact technique that Tomcat's JreMemoryLeakPreventionListener uses to prevent ClassLoader leaks. Yes, this looks like the way JDBC Pool does with the Pool Cleaner. I would go for this as a last resort. A couple of notes: 1. This won't work under a SecurityManager. If you need to operate under a SecurityManager, have a look at the JreMemoryLeakPreventionListener code and adapt it to the above. 2. If the Oracle driver launches the thread when the DataSource is created, it might happen too early for a ServletContextListener to intervene. In that case, simply modify the JreMEmoryLeakPreventionListener code directly. Patches are always welcome. Not, it is not. I have attached VisualVM to Tomcat VM and have seen that the thread is forked when the first query is executed. That's good news: it means that if you need it fixed *right away*, you don't need to hack-and-recompile Tomcat: you can just do it in your own ServletContextListener. 3. If Oracle fixes this bug, Tomcat should not prevent against it. If Oracle refuses to acknowledge/fix/etc. this bug, then it may make sense to include such code in JreMemoryLeakPreventionListener, but it should probably be done in such a way that a) it's not enabled by default and b) the query