After being plaged by PermGen space OutOfMemoryErrors I came across an http://java.dzone.com/articles/memory-leak-protection-tomcat interview with Mark Thomas announcing the solution for these kinds of memory leaks. I upgraded Tomcat 6.0.26 on Solaris 5.8 machine and repeatedly deployed and undeployed my applications. However, the problem remained. After many weeks, I think I got it:
Besides the enhanced WebappClassLoader the new Tomcat uses a Listener org.apache.catalina.core.JreMemoryLeakPreventionListener, that has to be included in the server.xml file. By upgrading, I missed that. The class javax.security.auth.Policy has a static member contextClassLoader that can refer to the WebappClassLoader. from jhat: <h3>Static reference from javax.security.auth.Policy.contextClassLoader<small> (from ../object/0xdc035580 class javax.security.auth.Policy) </small> :</h3>--> ../object/0xeb5a84a0 org.apache.catalina.loader.webappclassloa...@0xeb5a84a0 (157 bytes) <br> The class sun.rmi.server.LoaderHander has a static member LoaderTable that can contain the WebappClassLoader when using RMI. >From jhat: <h3>Static reference from sun.rmi.server.LoaderHandler.loaderTable<small> (from ../object/0xdbf2b258 class sun.rmi.server.LoaderHandler) </small> :</h3>--> ../object/0xeb7b52d0 java.util.hash...@0xeb7b52d0 (40 bytes) (field table:)<br> --> ../object/0xeb7b52f8 [Ljava.util.HashMap$Entry;@0xeb7b52f8 (40 bytes) (Element 0 of [Ljava.util.HashMap$Entry;@0xeb7b52f8:)<br> --> ../object/0xeb7b5388 java.util.hashmap$en...@0xeb7b5388 (24 bytes) (field key:)<br> --> ../object/0xeb7b53a0 sun.rmi.server.loaderhandler$loader...@0xeb7b53a0 (20 bytes) (field parent:)<br> --> ../object/0xeb5a84a0 org.apache.catalina.loader.webappclassloa...@0xeb5a84a0 (157 bytes) <br> The clearReferencesThreadLocals() method of WebappClassLoader clears two Thread tables from references to the WebappClassLoader. When using axis, the tables can have a HashMap as key, which contains (possibly after many nested HashMaps) a class that was loaded by the WebappClassLoader. >From jhat: <h3>Static reference from org.apache.catalina.ServerFactory.server<small> (from ../object/0xdba00178 class org.apache.catalina.ServerFactory) </small> :</h3>--> ../object/0xeb400668 org.apache.catalina.core.standardser...@0xeb400668 (67 bytes) (field services:)<br> --> ../object/0xeb4b4178 [Lorg.apache.catalina.Service;@0xeb4b4178 (12 bytes) (Element 0 of [Lorg.apache.catalina.Service;@0xeb4b4178:)<br> --> ../object/0xeb4b4188 org.apache.catalina.core.standardserv...@0xeb4b4188 (62 bytes) (field container:)<br> --> ../object/0xeb4b42c8 org.apache.catalina.core.standardeng...@0xeb4b42c8 (129 bytes) (field thread:)<br> --> ../object/0xeb58b6e0 java.lang.thr...@0xeb58b6e0 (104 bytes) (field threadLocals:)<br> --> ../object/0xeb58b800 java.lang.threadlocal$threadlocal...@0xeb58b800 (20 bytes) (field table:)<br> --> ../object/0xeb58b818 [Ljava.lang.ThreadLocal$ThreadLocalMap$Entry;@0xeb58b818 (72 bytes) (Element 8 of [Ljava.lang.ThreadLocal$ThreadLocalMap$Entry;@0xeb58b818:)<br> --> ../object/0xeb58b908 java.lang.threadlocal$threadlocalmap$en...@0xeb58b908 (28 bytes) (field value:)<br> --> ../object/0xeb58b928 java.util.hash...@0xeb58b928 (40 bytes) (field table:)<br> --> ../object/0xeb58b950 [Ljava.util.HashMap$Entry;@0xeb58b950 (72 bytes) (Element 0 of [Ljava.util.HashMap$Entry;@0xeb58b950:)<br> --> ../object/0xeb58bb40 java.util.hashmap$en...@0xeb58bb40 (24 bytes) (field key:)<br> --> ../object/0xdbe798a0 class org.apache.axis.types.Notation (84 bytes) (??:)<br> --> ../object/0xeb5a84a0 org.apache.catalina.loader.webappclassloa...@0xeb5a84a0 (157 bytes) <br> Suppose there is an web application A that uses a database and registers a Driver with the java.sql.DriverManager, and a web application B that doesn't use a database but has a jar file in its WEB-INF/lib directory that contains the same Driver. Suppose you unload webapplication B. Running the org.apache.catalina.loader.JdbcLeakPrevention class will actually register the Driver and leave it loaded! The cause is the way the DriverManager checks whether a ClassLoader has permission to load the Driver. It does that by calling ClassForName with the ClassLoader, which will load the class if the class has not been loaded by that ClassLoader. And loading a Driver triggers the Driver to register itself. Seems that the loop in org.apache.catalina.loader.JdbcLeakPrevention has to be run twice. I included the org.apache.catalina.core.JreMemoryLeakPreventionListener in my server.xml file. In the destroy method of my servlet, I did the following: <ul> <li>Set javax.security.auth.Policy.contextClassLoader to null if it referenced the WebappClassLoader.</li> <li>Iterates over the sun.rmi.server.LoaderHandler$LoaderKey items in sun.rmi.server.LoaderHandler, removing items whose parent field referred to the WebappClassLoader.</li> <li>Traverse Collections and Maps in the two java.lang.ThreadLocal table fields, clearing all deep references to the WebappClassLoader.</li> </ul> After that, my classes were unloaded after undeploy. Arjen -- View this message in context: http://old.nabble.com/More-sources-of-Tomcat-memory-leaks-tp29264214p29264214.html Sent from the Tomcat - Dev mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
