On 19/04/2016 16:25, Roger Riggs wrote: > Hi Mark, > > It may take a bit of time to unwind and find a good solution.
Hi Roger, Agreed. Your response has actually been a huge help. It gave me a few more things to look at and I have been able break the problem down significantly. I need to work on this some more before I come back with a much clearer statement of the problem. I hope to be in a position to do that later today or tomorrow. I'll probably do that in a new thread. Thanks for the pointers. Mark > > On 4/18/16 11:58 AM, Mark Thomas wrote: >> Hi, >> >> The Apache Tomcat community was asked by Rory O'Donnell for feedback on >> JDK 9 + Project Jigsaw. Having provided that feedback we were directed >> here so I have reproduced that feedback below. >> >> >> I've started testing Tomcat trunk with JDK 9 + Project Jigsaw and it >> looks like we are going to hit a bunch of problems related to Tomcat's >> memory leak protection. >> >> The short version is there are lots of JDK calls that can result in a >> reference being retained to the current class loader. If that class >> loader is the web application class loader it often ends up being pinned >> in memory. This triggers a memory leak when the web application is >> reloaded since the web application class loader is not eligible for GC. >> >> Tomcat generally uses reflection to find these problems. It then does >> one of two things: >> - If the JRE provides an API the application developer should have used >> to clean up the reference, Tomcat does this for them and then logs a >> very loud error message telling the developer they need to fix their app. >> - If there is nothing the developer could have done to avoid the >> problem, Tomcat cleans it up. Usually this is via reflection again. >> >> Reporting this second class of issues as JRE bugs has been on my TODO >> list for a long time. It looks like Java 9 is going to bump this to the >> top of the list. >> >> The first problem we have hit is related to RMI. The memory leak is >> triggered by a call to: >> java.rmi.registry.Registry.bind() or >> java.rmi.registry.Registry.rebind() >> >> The problem occurs when an object of a class loaded by the web >> application class loader is bound to the RMI registry. >> >> There is no standard API (that I have found) that completely removes all >> references. In particular >> java.rmi.registry.Registry.unbind() >> doesn't help. > unbind just removes the mapping from name to an exported object; > it does not un-export the object. > Unless it was exported as permanent, the normal GC/distributed GC should > clear it. > > I would think the normal application shutdown should both remove the > binding > and unexport it (perhaps forcibly). > > But you are working on the case where the application doesn't behave > properly. > > If you had a way to discover the exported remote objects, the normal > forcible unexport mechanism > should be sufficient. Handing out references to exports objects would > need some kind of security > check. > > It might be worth investigating an official shutdown mechanism though > linking it to a classloader > seems a bit special purpose. > > $.02, Roger > >> The code Tomcat uses to clean this up is at [1]. Essentially, we need to >> remove any reference to the web application's class loader from the RMI >> caches. >> >> With JDK 9 this fails with: >> java.lang.reflect.InaccessibleObjectException: Unable to make member of >> class sun.rmi.transport.Target accessible: module java.rmi does not >> export sun.rmi.transport to unnamed module ... >> >> I took a look at the JDK 9 API but I could not find a way to bypass >> this. >> >> Possible solutions include: >> 1. Some way that allows us to continue to use reflection as per the >> current code. >> >> 2. A new method somewhere in the RMI API that clears all references >> associated with a given class loader from the cache. >> >> 3. Modify Registry.unbind() so it removes all references. >> >> 4. Something else... >> >> I do have a concern with 3 on its own that, while that would allow >> applications to clear their own references, it would mean Tomcat would >> have no way to check for the problem. >> >> Ideally, I'd like to see the API extended so a) applications are able to >> clean up after themselves and b) Tomcat can check for leaked references >> and generate error messages for the ones found. >> >> Any and all suggestions gratefully received. >> >> Thanks, >> >> Mark >> >> [1] >> http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/catalina/loader/WebappClassLoaderBase.java?view=annotate#l2214 >> >
