Scott Ferguson wrote (2009-01-09 17:08): > On Jan 9, 2009, at 12:22 AM, Mattias Jiderhamn wrote: > > >> Scott Ferguson wrote (2008-11-26 16:53): >> >>> Thanks. >>> >>> Right now, our code base is a bit stuck due to the WebBeans/OSGi >>> upgrade (for Resin 4.0.0, was 3.2.2). Once that's cleaned up and I >>> can put up a snapshot I can take a look. >>> >> Scott Ferguson wrote (2008-12-29 20:41): >> >>> I've just made an early Resin 4.0 snapshot available. >>> >> Does this mean we can assume you will be able to look more closely at >> the memory leaks in 3.1 sometime soon...? >> > > I was hoping you could verify the 4.0 memory fixes before looking at > 3.1 Because several of those changes were more involved than I'm > comfortable with for 3.1, I'd prefer to make sure they work in 4.0 > first. > Seems to me nothing has changed in this regard in the 4.0 snapshot. I did my leak test as previously described, see below, and no classes were unloaded.
- Dowload http://jiderhamn.se/leak.war - Add some JARs to the WEB-INF/lib directory; preferrably a couple of large ones like spring.jar and hibernate.jar (don't use Resin JARs though). - Drop the WAR in a clean installation of Resin - Hit http://...:nn/leak (once is enough) - Force a redeploy by either deleting the webapps/leak dir or touch:ing leak.war - Hit http://...:nn/leak again - Repeat the last two steps for as long as you'd like /Mattias >>>> Now, here are the things really bugging me: >>>> 1. If I keep redeploying over and over, I will eventually get closer >>>> and closer to the Perm Gen Max (in some instances, I have seen the >>>> following behaivour instead turn up when Used reaches Init if Init >>>> is >>>> large enough). Then suddenly the unloadedClassCount is increased, >>>> but >>>> not with all the unused classes - only about the amount of one >>>> redeployment. Redeploy again, and it will increase another step. >>>> Meanwhile, the loadedClassCount remains pretty stable, since we are >>>> loading as many new classes as are unloaded. It's as if there was a >>>> FIFO queue/LRU cache of classloaders, so that the oldest one is >>>> garbage collected once there is not enough space for a new one. >>>> However, after a while there is (assumably) not enough space to >>>> create the new classloader before the old one is garbage collected, >>>> and I get OutOfMemoryError somewhere in the middle. Sometimes I am >>>> actually able to recover from this error by waiting for the GC to do >>>> it's job and then just try again. >>>> >>>> 2. Now I attached YourKit, looking for dangling classloaders as of >>>> the inital post. I found none. In fact, the Classes without >>>> Instances >>>> inspection only shows the classes in the added JARs from the last >>>> redeployment, so when tracing back to GC root, it goes via the >>>> current EnvironmentClassLoader which is correct. There are also no >>>> excessive instances of EnvironmentClassLoader. Hmm... Now wait a >>>> minute. Look at the total number of java.lang.Class objects. It does >>>> not match with the totalLoadedClassCount. In fact, the total number >>>> of classes found by YourKit is about the same as the >>>> totalLoadedClassCount on the very first hit of the application, >>>> before any redeployments. So from YourKits point of view, there is >>>> no >>>> classloader leak! But why then isn't the PermGen space reclaimed. >>>> This led me to wonder if there was some kind of JVM bug. (As a side >>>> note, I have yet to try with some other profiler) >>>> >>>> 3. So, I modified the test application (see commented out code in >>>> MyServlet.java) to load the classes of the JARs in a regular >>>> java.net.URLClassLoader which is then immediately thrown away. No >>>> leak. loadedClassCount is immediately decreased (and >>>> unloadedClassCount increased), as is the Used Perm Gen. That is, it >>>> behave the way we want the redeployment to behave. >>>> Ok, lets take one step down in the classloader hierachy and load all >>>> the classes via a disposable com.caucho.loader.DynamicClassLoader. >>>> No >>>> leak. >>>> So, what if I load them with a >>>> com.caucho.loader.EnvironmentClassLoader which is then destroy()ed >>>> and left for garbage collection. The leak is back! >>>> EnvironmentClassLoader has now applied to become prime suspect. In >>>> order to track things down I subclassed EnvironmentClassLoader >>>> inside >>>> my application and planned to make changes there. Well ... before >>>> any >>>> changes, the leak is nowhere to be found. >>>> >>>> What is going on here????? >>>> Anything that might help my sanity would be appreciated. >>>> >>>> /Mattias Jiderhamn >>>> >>>> >>>> Mattias Jiderhamn wrote (2008-11-05 06:43): >>>> >>>>> In support of this latest theory is the fact that YourKit shows two >>>>> GC roots for the HttpRequest. Apart from the >>>>> com.caucho.server.port.TcpConnection._request reference, the >>>>> request >>>>> is a root itself by being on the stack of a thread >>>>> (http--8080-...). >>>>> This could indicate a thread currently waiting in the >>>>> com.caucho.server.port.TcpConnection.run() method, where the >>>>> ServerRequest of the parent is also a local variable. >>>>> >>>>> Even if the request is reused, I believe the Invocation or the >>>>> ClassLoader of the invocation needs to be reset somehow, to release >>>>> the webapp classloader. >>>>> I can help try out a proposed fix to see if it solves the problem >>>>> (= >>>>> feel free to mail me off list). I might even give it a shot myself. >>>>> >>>>> /Mattias >>>>> >>
_______________________________________________ resin-interest mailing list [email protected] http://maillist.caucho.com/mailman/listinfo/resin-interest
