Hi Ilya, As Scott said, no, those are not lame questions. Apart of a Resin bug, it is unfortunately quite common for some libraries to be badly prepared to live in a "nenewing classloader" environment, so they leave references behind etc. And some of them cannot be fixed easily by de application developer. Off the top of my head, HSQDLB creates threads that are not killed unless you call a couple of methods explicitely, which does not happen in a generic Datasource environment, and various apache common libraries use ThreadLocal variables incorrectly so they keep copies of the old classes even when a context is reloaded.
The problem with singletons and threadlocals is not a problem with the technique per-se, but that they are often misused, hence creating such problems. The basic issue is that when an instance of a class loaded by a global classloader(ThreadLocal or any other class from the JDK or classes shared by all applications) keeps a reference to an instance of a class loaded by a web app classloader, then the instance is not GC-ed. The memory problem is not the instance itself, but because the instance is not GC-ed, the classloader that loaded it cannot be GC-ed and that means that all the libraries referenced by the classloader are not GC-ed themselves... etc. It's not an unsolvable problem, you "just" have to have all the libraries cleaning themselves correctly or at least providing methods for cleanup. But many of them were thought for standalone application usage, one classloader, no reloads... so they never thought about it. And there are way too many of the popular ones that misbehave. An ugly situation, all in all. But that's where "we live" today. S! D. Ilya Kasnacheev escribió: > I've got a weird problem for which I don't have enough knowlege. > > We've got a rich webapp (CMS) which uses servlet, xalan, hibernate, struts1, > a > few rhino, log4j. Right now I'm testing new version, old one was less > advanced and ran on resin 2. > > I've noticed that sometimes resin eats all allowed memory and refuses to work > anymore. After some heap dump digging I've found that when I restart web-app > by touching web.xml for example, resin creates new classloader and populates > it with classes and data, but old classloader doesn't go away, ever. > So after some development effort I would have around ten classloaders with > all > classes and stuff which would cause 'out of PermGen space' error. It looks > like this in jhat: > > select e._id.toString() from com.caucho.loader.EnvironmentClassLoader e > web-app:http://w3.uw.ru > host:http://w3.uw.ru > web-app:http://w3.uw.ru > web-app:http://w3.uw.ru > web-app:http://w3.uw.ru > web-app:http://w3.uw.ru > web-app:http://w3.uw.ru > web-app:http://w3.uw.ru > web-app:http://w3.uw.ru > cluster:app-tier > > After some more digging I've found that EnvironmentClassLoader is referenced > from apache-commons LogFactory.factories Hashtable. A-ha, thought I, that > must be why. I've added LogFactory.release() to the code that is supposed to > be executed when web-app is shut down. > > Running corrected version revealed that classloaders aren't still recycled, > while I've got rid of that Hashtable reference. Still standing references, > when I filter out all 'class Foo' and 'ProtectionDomain' lines, are like > this: > [EMAIL PROTECTED] (113 bytes) : field > _parentLoader > [EMAIL PROTECTED] (16 bytes) : field _loader > [EMAIL PROTECTED] (33 bytes) : field > _loader > [EMAIL PROTECTED] (159 bytes) : field > _owner > [EMAIL PROTECTED] (159 bytes) : field > _owner > [EMAIL PROTECTED] (64 bytes) : field > _classLoader > [EMAIL PROTECTED] (48 bytes) : field loader > ...about 30 Package's more... > [EMAIL PROTECTED] (48 bytes) : field loader > [EMAIL PROTECTED] (24 bytes) : field referent > [EMAIL PROTECTED] (36 bytes) : field referent > [EMAIL PROTECTED] (36 bytes) : field referent > [EMAIL PROTECTED] (36 bytes) : field referent > [EMAIL PROTECTED] (50 bytes) : field parent > > I wonder if there is something which keeps classloader from being marked as > garbage. All com.caucho objects as I understand are app-wide and aren't > referenced from outside, I would check that but I'm not sure I'm going the > right way. > > So my questions are: > Why might that all happen? > How would I debug the specific reason of my problem? How to spot it? > Are there anything else interesting about classloaders that I should better > know? > Maybe my problem isn't related to ClassLoader directly, maybe something else > gets lost in cache somewhere and never dereferenced? Any clues for that case? > > P.S. Wiki say: If the error occurs only after the redeployment and restart of > new applications, then the likely cause is that the JVM cannot garbage > collect old classes that are replaced because there are references to the old > classes, as discussed in [[Classloader references]]. > Where there is no such article. Where is that classloader reference, if it > exists at all? > > I would appreciate any help. > > P.P.S. Unrelated directly, but still interesting. > "Common application errors include: > Singleton or static hash maps and caches, esp check for clearing web-app > restarts. > web-app variables (like the "application (variable)" variable), stored in a > static member of a class > ThreadLocal variables that are not properly cleared at the end of each > request." > Aren't singletons and static hash maps referenced from webapp classes are > supposed to be dropped completely, reloaded and repopulated when web-app > restarts? > Why would not I store web-app variables as static members of webapp classes, > if those classes are supposed to be dropped-and-reread when web-app restarts > anyway? > What's wrong with static infinite-lifetime ThreadLocal variables? I thought > there would be one per thread, and if I have something like ten threads > under that classloader, I think ten unmutable variables aren't going to eat > up all memory even if they're never destroyed. > > P.P.P.S. Sorry for my possibly lame questions. > > > _______________________________________________ > resin-interest mailing list > resin-interest@caucho.com > http://maillist.caucho.com/mailman/listinfo/resin-interest -- ------------------------------------------- Daniel Lopez Janariz ([EMAIL PROTECTED]) Web Services Centre for Information and Technology Balearic Islands University (SPAIN) ------------------------------------------- _______________________________________________ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
