I'm glad to see there's someone else with these problems. I've had  
problems with leaks since 3.0. I rely on resin to re-load my webapp  
every time I rebuild a class, and it will do that a few times before  
getting a PermGen exception, at which point I have to kill and restart  
resin. I'm on 3.2.1 now, but won't be moving to 4.0 for a bit yet.

On Mar 25, 2009, at 07:35:32, Mattias Jiderhamn wrote:

> Scott Ferguson wrote (2009-01-09 17:08):
>> On Jan 9, 2009, at 12:22 AM, Mattias Jiderhamn wrote:
>>> Scott Ferguson wrote (2008-11-26 16:53):
>>>> Thanks.
>>>> Right now, our code base is a bit stuck due to the WebBeans/OSGi
>>>> upgrade (for Resin 4.0.0, was 3.2.2).  Once that's cleaned up and I
>>>> can put up a snapshot I can take a look.
>>> Scott Ferguson wrote (2008-12-29 20:41):
>>>> I've just made an early Resin 4.0 snapshot available.
>>> Does this mean we can assume you will be able to look more closely  
>>> at
>>> the memory leaks in 3.1 sometime soon...?
>> I was hoping you could verify the 4.0 memory fixes before looking at
>> 3.1  Because several of those changes were more involved than I'm
>> comfortable with for 3.1, I'd prefer to make sure they work in 4.0
>> first.
> Seems to me nothing has changed in this regard in the 4.0 snapshot.  
> I did my leak test as previously described, see below, and no  
> classes were unloaded.
> - Dowload http://jiderhamn.se/leak.war
> - Add some JARs to the WEB-INF/lib directory; preferrably a couple  
> of large ones like spring.jar and hibernate.jar (don't use Resin  
> JARs though).
> - Drop the WAR in a clean installation of Resin
> - Hit http://...:nn/leak (once is enough)
> - Force a redeploy by either deleting the webapps/leak dir or  
> touch:ing leak.war
> - Hit http://...:nn/leak again
> - Repeat the last two steps for as long as you'd like
>  /Mattias
>>>>> Now, here are the things really bugging me:
>>>>> 1. If I keep redeploying over and over, I will eventually get  
>>>>> closer
>>>>> and closer to the Perm Gen Max (in some instances, I have seen the
>>>>> following behaivour instead turn up when Used reaches Init if Init
>>>>> is
>>>>> large enough). Then suddenly the unloadedClassCount is increased,
>>>>> but
>>>>> not with all the unused classes - only about the amount of one
>>>>> redeployment. Redeploy again, and it will increase another step.
>>>>> Meanwhile, the loadedClassCount remains pretty stable, since we  
>>>>> are
>>>>> loading as many new classes as are unloaded. It's as if there  
>>>>> was a
>>>>> FIFO queue/LRU cache of classloaders, so that the oldest one is
>>>>> garbage collected once there is not enough space for a new one.
>>>>> However, after a while there is (assumably) not enough space to
>>>>> create the new classloader before the old one is garbage  
>>>>> collected,
>>>>> and I get OutOfMemoryError somewhere in the middle. Sometimes I am
>>>>> actually able to recover from this error by waiting for the GC  
>>>>> to do
>>>>> it's job and then just try again.
>>>>> 2. Now I attached YourKit, looking for dangling classloaders as of
>>>>> the inital post. I found none. In fact, the Classes without
>>>>> Instances
>>>>> inspection only shows the classes in the added JARs from the last
>>>>> redeployment, so when tracing back to GC root, it goes via the
>>>>> current EnvironmentClassLoader which is correct. There are also no
>>>>> excessive instances of EnvironmentClassLoader. Hmm... Now wait a
>>>>> minute. Look at the total number of java.lang.Class objects. It  
>>>>> does
>>>>> not match with the totalLoadedClassCount. In fact, the total  
>>>>> number
>>>>> of classes found by YourKit is about the same as the
>>>>> totalLoadedClassCount on the very first hit of the application,
>>>>> before any redeployments. So from YourKits point of view, there is
>>>>> no
>>>>> classloader leak! But why then isn't the PermGen space reclaimed.
>>>>> This led me to wonder if there was some kind of JVM bug. (As a  
>>>>> side
>>>>> note, I have yet to try with some other profiler)
>>>>> 3. So, I modified the test application (see commented out code in
>>>>> MyServlet.java) to load the classes of the JARs in a regular
>>>>> java.net.URLClassLoader which is then immediately thrown away. No
>>>>> leak. loadedClassCount is immediately decreased (and
>>>>> unloadedClassCount increased), as is the Used Perm Gen. That is,  
>>>>> it
>>>>> behave the way we want the redeployment to behave.
>>>>> Ok, lets take one step down in the classloader hierachy and load  
>>>>> all
>>>>> the classes via a disposable com.caucho.loader.DynamicClassLoader.
>>>>> No
>>>>> leak.
>>>>> So, what if I load them with a
>>>>> com.caucho.loader.EnvironmentClassLoader which is then destroy()ed
>>>>> and left for garbage collection. The leak is back!
>>>>> EnvironmentClassLoader has now applied to become prime suspect. In
>>>>> order to track things down I subclassed EnvironmentClassLoader
>>>>> inside
>>>>> my application and planned to make changes there. Well ... before
>>>>> any
>>>>> changes, the leak is nowhere to be found.
>>>>> What is going on here?????
>>>>> Anything that might help my sanity would be appreciated.
>>>>> /Mattias Jiderhamn
>>>>> Mattias Jiderhamn wrote (2008-11-05 06:43):
>>>>>> In support of this latest theory is the fact that YourKit shows  
>>>>>> two
>>>>>> GC roots for the HttpRequest. Apart from the
>>>>>> com.caucho.server.port.TcpConnection._request reference, the
>>>>>> request
>>>>>> is a root itself by being on the stack of a thread
>>>>>> (http--8080-...).
>>>>>> This could indicate a thread currently waiting in the
>>>>>> com.caucho.server.port.TcpConnection.run() method, where the
>>>>>> ServerRequest of the parent is also a local variable.
>>>>>> Even if the request is reused, I believe the Invocation or the
>>>>>> ClassLoader of the invocation needs to be reset somehow, to  
>>>>>> release
>>>>>> the webapp classloader.
>>>>>> I can help try out a proposed fix to see if it solves the problem
>>>>>> (=
>>>>>> feel free to mail me off list). I might even give it a shot  
>>>>>> myself.
>>>>>> /Mattias
> _______________________________________________
> resin-interest mailing list
> resin-interest@caucho.com
> http://maillist.caucho.com/mailman/listinfo/resin-interest


resin-interest mailing list

Reply via email to