I've got a weird problem for which I don't have enough knowlege. We've got a rich webapp (CMS) which uses servlet, xalan, hibernate, struts1, a few rhino, log4j. Right now I'm testing new version, old one was less advanced and ran on resin 2.
I've noticed that sometimes resin eats all allowed memory and refuses to work anymore. After some heap dump digging I've found that when I restart web-app by touching web.xml for example, resin creates new classloader and populates it with classes and data, but old classloader doesn't go away, ever. So after some development effort I would have around ten classloaders with all classes and stuff which would cause 'out of PermGen space' error. It looks like this in jhat: select e._id.toString() from com.caucho.loader.EnvironmentClassLoader e web-app:http://w3.uw.ru host:http://w3.uw.ru web-app:http://w3.uw.ru web-app:http://w3.uw.ru web-app:http://w3.uw.ru web-app:http://w3.uw.ru web-app:http://w3.uw.ru web-app:http://w3.uw.ru web-app:http://w3.uw.ru cluster:app-tier After some more digging I've found that EnvironmentClassLoader is referenced from apache-commons LogFactory.factories Hashtable. A-ha, thought I, that must be why. I've added LogFactory.release() to the code that is supposed to be executed when web-app is shut down. Running corrected version revealed that classloaders aren't still recycled, while I've got rid of that Hashtable reference. Still standing references, when I filter out all 'class Foo' and 'ProtectionDomain' lines, are like this: [EMAIL PROTECTED] (113 bytes) : field _parentLoader [EMAIL PROTECTED] (16 bytes) : field _loader [EMAIL PROTECTED] (33 bytes) : field _loader [EMAIL PROTECTED] (159 bytes) : field _owner [EMAIL PROTECTED] (159 bytes) : field _owner [EMAIL PROTECTED] (64 bytes) : field _classLoader [EMAIL PROTECTED] (48 bytes) : field loader ...about 30 Package's more... [EMAIL PROTECTED] (48 bytes) : field loader [EMAIL PROTECTED] (24 bytes) : field referent [EMAIL PROTECTED] (36 bytes) : field referent [EMAIL PROTECTED] (36 bytes) : field referent [EMAIL PROTECTED] (36 bytes) : field referent [EMAIL PROTECTED] (50 bytes) : field parent I wonder if there is something which keeps classloader from being marked as garbage. All com.caucho objects as I understand are app-wide and aren't referenced from outside, I would check that but I'm not sure I'm going the right way. So my questions are: Why might that all happen? How would I debug the specific reason of my problem? How to spot it? Are there anything else interesting about classloaders that I should better know? Maybe my problem isn't related to ClassLoader directly, maybe something else gets lost in cache somewhere and never dereferenced? Any clues for that case? P.S. Wiki say: If the error occurs only after the redeployment and restart of new applications, then the likely cause is that the JVM cannot garbage collect old classes that are replaced because there are references to the old classes, as discussed in [[Classloader references]]. Where there is no such article. Where is that classloader reference, if it exists at all? I would appreciate any help. P.P.S. Unrelated directly, but still interesting. "Common application errors include: Singleton or static hash maps and caches, esp check for clearing web-app restarts. web-app variables (like the "application (variable)" variable), stored in a static member of a class ThreadLocal variables that are not properly cleared at the end of each request." Aren't singletons and static hash maps referenced from webapp classes are supposed to be dropped completely, reloaded and repopulated when web-app restarts? Why would not I store web-app variables as static members of webapp classes, if those classes are supposed to be dropped-and-reread when web-app restarts anyway? What's wrong with static infinite-lifetime ThreadLocal variables? I thought there would be one per thread, and if I have something like ten threads under that classloader, I think ten unmutable variables aren't going to eat up all memory even if they're never destroyed. P.P.P.S. Sorry for my possibly lame questions. _______________________________________________ resin-interest mailing list [email protected] http://maillist.caucho.com/mailman/listinfo/resin-interest
