I'm glad to see there's someone else with these problems. I've had problems with leaks since 3.0. I rely on resin to re-load my webapp every time I rebuild a class, and it will do that a few times before getting a PermGen exception, at which point I have to kill and restart resin. I'm on 3.2.1 now, but won't be moving to 4.0 for a bit yet.
On Mar 25, 2009, at 07:35:32, Mattias Jiderhamn wrote: > Scott Ferguson wrote (2009-01-09 17:08): >> >> On Jan 9, 2009, at 12:22 AM, Mattias Jiderhamn wrote: >> >> >>> Scott Ferguson wrote (2008-11-26 16:53): >>> >>>> Thanks. >>>> >>>> Right now, our code base is a bit stuck due to the WebBeans/OSGi >>>> upgrade (for Resin 4.0.0, was 3.2.2). Once that's cleaned up and I >>>> can put up a snapshot I can take a look. >>>> >>> Scott Ferguson wrote (2008-12-29 20:41): >>> >>>> I've just made an early Resin 4.0 snapshot available. >>>> >>> Does this mean we can assume you will be able to look more closely >>> at >>> the memory leaks in 3.1 sometime soon...? >>> >> >> I was hoping you could verify the 4.0 memory fixes before looking at >> 3.1 Because several of those changes were more involved than I'm >> comfortable with for 3.1, I'd prefer to make sure they work in 4.0 >> first. >> > Seems to me nothing has changed in this regard in the 4.0 snapshot. > I did my leak test as previously described, see below, and no > classes were unloaded. > > - Dowload http://jiderhamn.se/leak.war > - Add some JARs to the WEB-INF/lib directory; preferrably a couple > of large ones like spring.jar and hibernate.jar (don't use Resin > JARs though). > - Drop the WAR in a clean installation of Resin > - Hit http://...:nn/leak (once is enough) > - Force a redeploy by either deleting the webapps/leak dir or > touch:ing leak.war > - Hit http://...:nn/leak again > - Repeat the last two steps for as long as you'd like > > /Mattias > > > > > >>>>> Now, here are the things really bugging me: >>>>> 1. If I keep redeploying over and over, I will eventually get >>>>> closer >>>>> and closer to the Perm Gen Max (in some instances, I have seen the >>>>> following behaivour instead turn up when Used reaches Init if Init >>>>> is >>>>> large enough). Then suddenly the unloadedClassCount is increased, >>>>> but >>>>> not with all the unused classes - only about the amount of one >>>>> redeployment. Redeploy again, and it will increase another step. >>>>> Meanwhile, the loadedClassCount remains pretty stable, since we >>>>> are >>>>> loading as many new classes as are unloaded. It's as if there >>>>> was a >>>>> FIFO queue/LRU cache of classloaders, so that the oldest one is >>>>> garbage collected once there is not enough space for a new one. >>>>> However, after a while there is (assumably) not enough space to >>>>> create the new classloader before the old one is garbage >>>>> collected, >>>>> and I get OutOfMemoryError somewhere in the middle. Sometimes I am >>>>> actually able to recover from this error by waiting for the GC >>>>> to do >>>>> it's job and then just try again. >>>>> >>>>> 2. Now I attached YourKit, looking for dangling classloaders as of >>>>> the inital post. I found none. In fact, the Classes without >>>>> Instances >>>>> inspection only shows the classes in the added JARs from the last >>>>> redeployment, so when tracing back to GC root, it goes via the >>>>> current EnvironmentClassLoader which is correct. There are also no >>>>> excessive instances of EnvironmentClassLoader. Hmm... Now wait a >>>>> minute. Look at the total number of java.lang.Class objects. It >>>>> does >>>>> not match with the totalLoadedClassCount. In fact, the total >>>>> number >>>>> of classes found by YourKit is about the same as the >>>>> totalLoadedClassCount on the very first hit of the application, >>>>> before any redeployments. So from YourKits point of view, there is >>>>> no >>>>> classloader leak! But why then isn't the PermGen space reclaimed. >>>>> This led me to wonder if there was some kind of JVM bug. (As a >>>>> side >>>>> note, I have yet to try with some other profiler) >>>>> >>>>> 3. So, I modified the test application (see commented out code in >>>>> MyServlet.java) to load the classes of the JARs in a regular >>>>> java.net.URLClassLoader which is then immediately thrown away. No >>>>> leak. loadedClassCount is immediately decreased (and >>>>> unloadedClassCount increased), as is the Used Perm Gen. That is, >>>>> it >>>>> behave the way we want the redeployment to behave. >>>>> Ok, lets take one step down in the classloader hierachy and load >>>>> all >>>>> the classes via a disposable com.caucho.loader.DynamicClassLoader. >>>>> No >>>>> leak. >>>>> So, what if I load them with a >>>>> com.caucho.loader.EnvironmentClassLoader which is then destroy()ed >>>>> and left for garbage collection. The leak is back! >>>>> EnvironmentClassLoader has now applied to become prime suspect. In >>>>> order to track things down I subclassed EnvironmentClassLoader >>>>> inside >>>>> my application and planned to make changes there. Well ... before >>>>> any >>>>> changes, the leak is nowhere to be found. >>>>> >>>>> What is going on here????? >>>>> Anything that might help my sanity would be appreciated. >>>>> >>>>> /Mattias Jiderhamn >>>>> >>>>> >>>>> Mattias Jiderhamn wrote (2008-11-05 06:43): >>>>> >>>>>> In support of this latest theory is the fact that YourKit shows >>>>>> two >>>>>> GC roots for the HttpRequest. Apart from the >>>>>> com.caucho.server.port.TcpConnection._request reference, the >>>>>> request >>>>>> is a root itself by being on the stack of a thread >>>>>> (http--8080-...). >>>>>> This could indicate a thread currently waiting in the >>>>>> com.caucho.server.port.TcpConnection.run() method, where the >>>>>> ServerRequest of the parent is also a local variable. >>>>>> >>>>>> Even if the request is reused, I believe the Invocation or the >>>>>> ClassLoader of the invocation needs to be reset somehow, to >>>>>> release >>>>>> the webapp classloader. >>>>>> I can help try out a proposed fix to see if it solves the problem >>>>>> (= >>>>>> feel free to mail me off list). I might even give it a shot >>>>>> myself. >>>>>> >>>>>> /Mattias >>>>>> >>> > > _______________________________________________ > resin-interest mailing list > [email protected] > http://maillist.caucho.com/mailman/listinfo/resin-interest -- Rick _______________________________________________ resin-interest mailing list [email protected] http://maillist.caucho.com/mailman/listinfo/resin-interest
