Scott Ferguson wrote (2009-12-11 00:23):
Mattias Jiderhamn wrote:
So I've spent another day hunting that lo(ooo)ng standing PermGen memory leak in our application and/or Resin.


I made a new discovery which "shouldn't" be an issue, but could potentially fix problems. From my investigation it seems that whenever the application is reloaded, a reference to the old com.caucho.loader.EnvironmentClassLoader is kept in com.caucho.hessian.io.SerializerFactory._defaultFactory. I agree that in theory this shouldn't happen, since the ClassLoader is a weak reference key, /however/ if I add some code to the application shutdown, explicitly removing the EnvironmentClassLoader from _defaultFactory using reflection, the garbage collector is able to unload these classes.
I changed this in 4.0.2 (with some more changes in 4.0.3). Even though the key is a weak reference, the value is a strong reference.
Indeed, this has been fixed in the latest snapshot! Good job!
Sorry for not testing against that...


I should mention though, that there is still a minimum of two EnvironmentClassLoaders for the given application after reloading at least once. The former one seem to stick around somehow. We have discussed this before, Scott; how references are kept inside com.caucho.server.dispatch.Invocation, at least in a low traffic (dev / debugging) environment.
On a web-app change the Invocation cache is cleared, so there shouldn't be any old references there.
Could you please double check that? For example, what happens to the 503:ed requests received during redeployment? I have an examplifying YourKit (freely available EAP) snapshot I could send you if it helps.

Here is what I'm getting, which seems repeatable over and over in my environment:
Startup: 9k loaded classes, no unloaded classes, 49 MB PermGen
First redeployment: 15k loaded classes, no unloaded classes, 77 MB PermGen = one extra instance of the app with 6k classes loaded. Assumed a total of 2 classloaders Second redeployment: 15k loaded classes, 6k unloaded classes, 77 MB PermGen = initial/first(?) instance GC:ed and a new one loaded. Still 2 classloaders. Third redeployment: 21k loaded classes, 6k unloaded classes, 105 MB PermGen = one extra instance of the app with 6k classes loaded, nothing GC:ed. Assumed a total of 3 classloaders Fourth redeployment: 21k loaded classes, 12k unloaded classes, 105 MB PermGen = second(?) instance GC:ed and a new one loaded. Still 3 classloaders. (Fifth redeployment: 15k loaded classes, 24k unloaded classes, 77 MB PermGen = third and fourth(?) instance GC:ed and a new one loaded. 2 classloaders. Haven't repeated this test enough to build a thesis)


Not sure if this a real problem, but ideally the class loader of the previous version should be available for garbage collection before the classes of the new version are loaded ('cause there is no turning back anyway, is there...?).
In the past this could be a problem with JNI because a JNI library can only be loaded in one classloader (that may have been changed but it always was a JDK restriction.)
Are you saying this shouldn't be a issue in 4.0.2/snapshot running on JDK 1.6?

--

 </Mattias>

_______________________________________________
resin-interest mailing list
resin-interest@caucho.com
http://maillist.caucho.com/mailman/listinfo/resin-interest

Reply via email to