Aw: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
Gesendet: Freitag, 17. Mai 2013 um 14:26 Uhr Von: Mark Thomas ma...@apache.org On 17/05/2013 12:31, Michael-O wrote: Hi Mark, thanks again for the detailed answer, details inline. Gesendet: Freitag, 17. Mai 2013 um 11:36 Uhr Von: Mark Thomas ma...@apache.org An: Tomcat Users List users@tomcat.apache.org Betreff: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread On 17/05/2013 09:28, Michael-O wrote: Hi folks, there's now a follow-up on the issue [1]. Recap: JreMemoryLeakPreventionListener reported that my webapp has spawned OracleTimeoutPollingThread because I have set a QueryTimeout in the JDBC Pool. Mark Thomas noted that this is likely a bug in the driver. I have taken action and created a service request with Oracle. My personal analysis with VisualVM: The thread is spawned when the first query is run. The thread keeps running when the webapp is shut down or undeployed. Classes remain in memory. The JDBC Pool does a simple Class.forName to load the driver, it neither uses the DriverManager loading nor does it actively unload the driver. Oracle answer: They were able to reproduce the issue. The technical analysis says: Hi Michael, I confirmned internally that this message from Tomcat can be ignored as there is no risk of any real leak from OracleTimeoutPollingThread thread. This thread is related to the JDBC driver which may be used by many apps simultaneously. Unloading the app does not unload the driver and should not and does not stop the thread. It seems to be the design behaviour. So my questions would be: 1. Is that still not a false positive? No, that is not a false positive. The response from Oracle is wrong. There is nothing wrong with the driver creating the thread or the thread continuing after the web application has stopped. The problem is as follows: 1. When the JDBC driver creates the Thread, the Thread's context class loader (as returned by Thread.getContextClassLoader()) is set to the web application's class loader. 2. The correct behaviour at this point would be for the Driver to set the Thread's context class loader to the class loader that loaded the Driver class when the Thread is created. 3. The memory leak occurs as follows: - the web application is stopped - Tomcat clears all references to the web application and its classes - The web application should be eligible for garbage collection - The JDBC driver is still loaded (as it should be) - The JDBC driver retains a reference to the Thread (as it should) - The thread retains a reference to the web application class loader (this is the memory leak). The reference chain is: a) JDBC driver b) Thread c) Web application class loader d) Every class loaded by the web app Everything from c) onwards should be eligible for garbage collection but isn't because of the leak in the Thread that is retaining a reference to the web application class loader. This is what I would assume as correct behavior. If the thread is still attached ot the WebAppClassLoader, does that mean that the WebAppClassLoader cannot be garbage collected? The correct behaviour is that there is no reference to c) held by b) and therefore c) onwards would be eligible for GC (assuming the web app has been stopped). The problem here is that because b) has a reference to c), c) can't be GC'd and that is a memory leak. Exactly, WebAppClassLoader keeps in a stale status. 2. Why does the JDBC Pool not unload the driver? That my cause the thread to stop after the last app has been stopped/undeployed. 1. Because the driver is in CATALINA_HOME/lib it is available to other applications. So first app, loads driver and keeps it in memory. Even if all apps are undeployed, driver remains cached and GCed when Tomcat is shut down? The JDBC driver will be loaded though the services API as soon as any code references the DriverManager. 2. JDBC drivers are never automatically unloaded. You have to do so explicitly (not doing so is an other source of memory leaks when the driver is packaged in WEB-INF/lib). OK, this won't be the case. Driver is always shared here. You need to go back to Oracle. Yes, I will. We're paying probably a lot of money for he company-wide license. I'd expect so. From my own experience with Oracle commercial support for things Java related, you'll have to go through the This is a bug. No it isn't. Yes it is... cycle several times before you manage to get the issue in front of someone with the necessary knowledge and skills to correctly identify this is a bug in Oracle's JDBC driver. I'd recommend escalating it through your account manager sooner rather than later. Feel free to point Oracle support to this thread and/or my
Aw: RE: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread
-Original Message- From: Mark Thomas [mailto:ma...@apache.org] Sent: Friday, May 17, 2013 7:26 AM To: Tomcat Users List Subject: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread On 17/05/2013 12:31, Michael-O wrote: Hi Mark, thanks again for the detailed answer, details inline. Gesendet: Freitag, 17. Mai 2013 um 11:36 Uhr Von: Mark Thomas ma...@apache.org An: Tomcat Users List users@tomcat.apache.org Betreff: Re: Follow-up: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool and OracleTimeoutPollingThread On 17/05/2013 09:28, Michael-O wrote: Hi folks, there's now a follow-up on the issue [1]. Recap: JreMemoryLeakPreventionListener reported that my webapp has spawned OracleTimeoutPollingThread because I have set a QueryTimeout in the JDBC Pool. Mark Thomas noted that this is likely a bug in the driver. I have taken action and created a service request with Oracle. My personal analysis with VisualVM: The thread is spawned when the first query is run. The thread keeps running when the webapp is shut down or undeployed. Classes remain in memory. The JDBC Pool does a simple Class.forName to load the driver, it neither uses the DriverManager loading nor does it actively unload the driver. Oracle answer: They were able to reproduce the issue. The technical analysis says: Hi Michael, I confirmned internally that this message from Tomcat can be ignored as there is no risk of any real leak from OracleTimeoutPollingThread thread. This thread is related to the JDBC driver which may be used by many apps simultaneously. Unloading the app does not unload the driver and should not and does not stop the thread. It seems to be the design behaviour. So my questions would be: 1. Is that still not a false positive? No, that is not a false positive. The response from Oracle is wrong. There is nothing wrong with the driver creating the thread or the thread continuing after the web application has stopped. The problem is as follows: 1. When the JDBC driver creates the Thread, the Thread's context class loader (as returned by Thread.getContextClassLoader()) is set to the web application's class loader. 2. The correct behaviour at this point would be for the Driver to set the Thread's context class loader to the class loader that loaded the Driver class when the Thread is created. 3. The memory leak occurs as follows: - the web application is stopped - Tomcat clears all references to the web application and its classes - The web application should be eligible for garbage collection - The JDBC driver is still loaded (as it should be) - The JDBC driver retains a reference to the Thread (as it should) - The thread retains a reference to the web application class loader (this is the memory leak). The reference chain is: a) JDBC driver b) Thread c) Web application class loader d) Every class loaded by the web app Everything from c) onwards should be eligible for garbage collection but isn't because of the leak in the Thread that is retaining a reference to the web application class loader. This is what I would assume as correct behavior. If the thread is still attached ot the WebAppClassLoader, does that mean that the WebAppClassLoader cannot be garbage collected? The correct behaviour is that there is no reference to c) held by b) and therefore c) onwards would be eligible for GC (assuming the web app has been stopped). The problem here is that because b) has a reference to c), c) can't be GC'd and that is a memory leak. 2. Why does the JDBC Pool not unload the driver? That my cause the thread to stop after the last app has been stopped/undeployed. 1. Because the driver is in CATALINA_HOME/lib it is available to other applications. So first app, loads driver and keeps it in memory. Even if all apps are undeployed, driver remains cached and GCed when Tomcat is shut down? The JDBC driver will be loaded though the services API as soon as any code references the DriverManager. 2. JDBC drivers are never automatically unloaded. You have to do so explicitly (not doing so is an other source of memory leaks when the driver is packaged in WEB-INF/lib). OK, this won't be the case. Driver is always shared here. You need to go back to Oracle. Yes, I will. We're paying probably a lot of money for he company-wide license. I'd expect so. From my own experience with Oracle commercial support for things Java related, you'll have to go through the This is a bug. No it isn't. Yes it is... cycle several times before you manage to get the issue in front of someone with the
