Scott Ferguson wrote (2009-01-09 17:08):
> On Jan 9, 2009, at 12:22 AM, Mattias Jiderhamn wrote:
>> Scott Ferguson wrote (2008-11-26 16:53):
>>> Thanks.
>>> Right now, our code base is a bit stuck due to the WebBeans/OSGi
>>> upgrade (for Resin 4.0.0, was 3.2.2).  Once that's cleaned up and I
>>> can put up a snapshot I can take a look.
>> Scott Ferguson wrote (2008-12-29 20:41):
>>> I've just made an early Resin 4.0 snapshot available.
>> Does this mean we can assume you will be able to look more closely at
>> the memory leaks in 3.1 sometime soon...?
> I was hoping you could verify the 4.0 memory fixes before looking at  
> 3.1  Because several of those changes were more involved than I'm  
> comfortable with for 3.1, I'd prefer to make sure they work in 4.0  
> first.
Seems to me nothing has changed in this regard in the 4.0 snapshot. I
did my leak test as previously described, see below, and no classes were

- Dowload
- Add some JARs to the WEB-INF/lib directory; preferrably a couple of
large ones like spring.jar and hibernate.jar (don't use Resin JARs though).
- Drop the WAR in a clean installation of Resin
- Hit http://...:nn/leak (once is enough)
- Force a redeploy by either deleting the webapps/leak dir or touch:ing
- Hit http://...:nn/leak again
- Repeat the last two steps for as long as you'd like


>>>> Now, here are the things really bugging me:
>>>> 1. If I keep redeploying over and over, I will eventually get closer
>>>> and closer to the Perm Gen Max (in some instances, I have seen the
>>>> following behaivour instead turn up when Used reaches Init if Init  
>>>> is
>>>> large enough). Then suddenly the unloadedClassCount is increased,  
>>>> but
>>>> not with all the unused classes - only about the amount of one
>>>> redeployment. Redeploy again, and it will increase another step.
>>>> Meanwhile, the loadedClassCount remains pretty stable, since we are
>>>> loading as many new classes as are unloaded. It's as if there was a
>>>> FIFO queue/LRU cache of classloaders, so that the oldest one is
>>>> garbage collected once there is not enough space for a new one.
>>>> However, after a while there is (assumably) not enough space to
>>>> create the new classloader before the old one is garbage collected,
>>>> and I get OutOfMemoryError somewhere in the middle. Sometimes I am
>>>> actually able to recover from this error by waiting for the GC to do
>>>> it's job and then just try again.
>>>> 2. Now I attached YourKit, looking for dangling classloaders as of
>>>> the inital post. I found none. In fact, the Classes without  
>>>> Instances
>>>> inspection only shows the classes in the added JARs from the last
>>>> redeployment, so when tracing back to GC root, it goes via the
>>>> current EnvironmentClassLoader which is correct. There are also no
>>>> excessive instances of EnvironmentClassLoader. Hmm... Now wait a
>>>> minute. Look at the total number of java.lang.Class objects. It does
>>>> not match with the totalLoadedClassCount. In fact, the total number
>>>> of classes found by YourKit is about the same as the
>>>> totalLoadedClassCount on the very first hit of the application,
>>>> before any redeployments. So from YourKits point of view, there is  
>>>> no
>>>> classloader leak! But why then isn't the PermGen space reclaimed.
>>>> This led me to wonder if there was some kind of JVM bug. (As a side
>>>> note, I have yet to try with some other profiler)
>>>> 3. So, I modified the test application (see commented out code in
>>>> to load the classes of the JARs in a regular
>>>> which is then immediately thrown away. No
>>>> leak. loadedClassCount is immediately decreased (and
>>>> unloadedClassCount increased), as is the Used Perm Gen. That is, it
>>>> behave the way we want the redeployment to behave.
>>>> Ok, lets take one step down in the classloader hierachy and load all
>>>> the classes via a disposable com.caucho.loader.DynamicClassLoader.  
>>>> No
>>>> leak.
>>>> So, what if I load them with a
>>>> com.caucho.loader.EnvironmentClassLoader which is then destroy()ed
>>>> and left for garbage collection. The leak is back!
>>>> EnvironmentClassLoader has now applied to become prime suspect. In
>>>> order to track things down I subclassed EnvironmentClassLoader  
>>>> inside
>>>> my application and planned to make changes there. Well ... before  
>>>> any
>>>> changes, the leak is nowhere to be found.
>>>> What is going on here?????
>>>> Anything that might help my sanity would be appreciated.
>>>> /Mattias Jiderhamn
>>>> Mattias Jiderhamn wrote (2008-11-05 06:43):
>>>>> In support of this latest theory is the fact that YourKit shows two
>>>>> GC roots for the HttpRequest. Apart from the
>>>>> com.caucho.server.port.TcpConnection._request reference, the  
>>>>> request
>>>>> is a root itself by being on the stack of a thread  
>>>>> (http--8080-...).
>>>>> This could indicate a thread currently waiting in the
>>>>> method, where the
>>>>> ServerRequest of the parent is also a local variable.
>>>>> Even if the request is reused, I believe the Invocation or the
>>>>> ClassLoader of the invocation needs to be reset somehow, to release
>>>>> the webapp classloader.
>>>>> I can help try out a proposed fix to see if it solves the problem  
>>>>> (=
>>>>> feel free to mail me off list). I might even give it a shot myself.
>>>>> /Mattias

resin-interest mailing list

Reply via email to