-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Elias,
On 4/4/14, 11:03 AM, Elias Kopsiaftis wrote: > Ive done some more research into this problem with my co developer, > and we have a question. Is it possible that memory leaks would > cause tomcat to crash? Yes, but you'll likely get an OutOfMemoryError in your log, and then everything will go crazy. Some requests will work, others will not. Things will get slow. For us, the background thread often quits and sessions never expire, exacerbating the problem. But it won't happen silently (you may have to look for it) and it won't spontaneously undeploy and redeploy your web application. > Silly question I know, but heres technically what the situation > is. > > We did not add code to shut down all threads when application is > undeployed through tomcat manager. Tomcat will do this itself: you don't need to add any code. Or did you mean that you have Threads created by your application that continue to tun after your web app has been shut down? > When undeploying and redeploying we cause memory leaks due to our > application. Yes, this can happen. The Threads cause the WebappClassLoader to stay in memory, which means that all Class objects for the previous deployment are still hanging around. It can be several tens of MiB per reload. You should read this excellent presentation which explains the problem(s) and the solution(s): http://people.apache.org/~markt/presentations/2010-11-04-Memory-Leaks-60mins.pdf > Now, I guess the real question is, we seem to always deploy it > fine, but randomly at some point during the night it will crash, > and we only know that from checking next morning. What do you mean when you say "crash"? Exception? Spontaneous re-deployment? OutOfMemory? JVM crash (process exits)? > We have nothing logged that indicates why. If the process is gone without any indication of why (no shutdown message, no nothing), my only guess is Linux OOM-killer. You never actually came back to describe the environment you are in after saying you'd post that information "later today." If you're not on Linux.... I'm not sure. > Would memory leaks cause it to crash randomly even though nothing > is trying to connect to it, yet it would seem to deploy fine? I've never seen an OOME actually cause a JVM to quit. It just slogs onward, often not able to accomplish any useful work, but the process stays running. > Could that be the issue we are seeing? Is there any specific class > to debug in logging.properties that might indicate whats going on? > I added that line you guys mentioned about setting log level to > DEBUG from that class loader class, but it hasnt crashed since > then.... Describe what you mean by "crash", including as much details about it, and definitely read the presentation referenced above. Also definitely fix your web application to shut down its Threads. Whatever component launches them (e.g. ServletContextListener, Servlet.init, etc.) should stop them in the appropriate callback method (ServletContextListener.contextDestroyed, Servlet.destroy, etc.). - -chris > On Thu, Apr 3, 2014 at 12:06 PM, Christopher Schultz < > ch...@christopherschultz.net> wrote: > > Mark, > > On 4/2/14, 5:20 PM, Mark Eggers wrote: >>>> Chris, >>>> >>>> On 4/2/2014 1:54 PM, Christopher Schultz wrote: >>>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 >>>>> >>>>> Mark, >>>>> >>>>> On 4/2/14, 4:30 PM, Mark Eggers wrote: >>>>>> Chris, >>>>>> >>>>>> On 4/2/2014 1:05 PM, Christopher Schultz wrote: >>>>>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 >>>>>>> >>>>>>> Chuck, >>>>>>> >>>>>>> On 4/2/14, 8:21 AM, Caldarale, Charles R wrote: >>>>>>>>> From: Elias Kopsiaftis [mailto:yemi...@gmail.com] >>>>>>>>> Subject: tomcat randomly undeploys and redeploys >>>>>>>>> the applications >>>>>>>> >>>>>>>>> I deploy the application, then in the log file >>>>>>>>> catalina.out i get many messages from >>>>>>>>> WebappClassLoader clearReferencesThreads saying >>>>>>>>> threads appear to have started but have failed to >>>>>>>>> stop >>>>>>>> >>>>>>>> This is an indication that your webapp is not >>>>>>>> managing its threads properly. >>>>>>>> >>>>>>>>> then finally, Ill get a message from HostConfig >>>>>>>>> checkResources that says its undeploying the >>>>>>>>> context, and then it redeploys. >>>>>>>> >>>>>>>> This is sometimes caused by incorrect timestamps on >>>>>>>> the bits of the webapp that Tomcat monitors, or an >>>>>>>> incorrect clock setting on the system Tomcat is >>>>>>>> running on. The mismatch makes it appear that the >>>>>>>> webapp is being updated continuously. >>>>>>> >>>>>>> I've found that in development, a single update can >>>>>>> cause Tomcat to go into a loop of redeployments, >>>>>>> re-deploying my web application every few seconds or >>>>>>> so. If I let it go, it does finally stop reloading and >>>>>>> settle down. >>>>>>> >>>>>> >>>>>> Can you describe your development environment a little >>>>>> bit, and any thoughts as to what might trigger this loop >>>>>> of redeployments? >>>>> >>>>> I use Eclipse for development, but our "real" build process >>>>> is ant-based. We have some watched resources configured >>>>> outside the default (stuff like Struts config files, >>>>> etc.). >>>>> >>>> >>>> Ah, makes sense. >>>> >>>>> When I do a build while Tomcat is running, usually I get >>>>> one webapp reload, but sometimes I get a series of reloads. >>>>> It usually gets so irritating (our webapp takes about 10 >>>>> seconds to reload) that I just kill Tomcat and immediately >>>>> restart it. It starts up once and all is well after that. >>>>> >>>> >>>> Yep, and in the process more files are copied about, and >>>> that triggers another reload. >>>> >>>> Fun, fun. > > No, the deployment update takes like one or two seconds. It's > usually something like copying less than 10 class files or > whatever. It's nearly instantaneous. Whatever happens, it's not > because I'm updating files during the reload. I could understand > that situation. > > What I observe is that I update my application, I wait maybe 10 > seconds, and then Tomcat reloads my application multiple times > before I just kill it. > >>>>>> I've not seen this, but it could explain some issues some >>>>>> the developers I support are seeing. >>>>> >>>>> It definitely happens, and I never remember to enable the >>>>> DEBUG logging to find out what resource it thinks has been >>>>> updated until after it happens, at which point I just don't >>>>> care. Perhaps I should enable it right now :) >>>>> >>>>> - -chris >>>> >>>> I've managed to make this happen in my environment now >>>> (NetBeans 7.4, Maven 3.2.1, Tomcat 7.0.42 - all will be >>>> upgraded soon). I just needed an application that takes a bit >>>> longer to load. I only managed to trigger two reloads, so >>>> it's not much of an issue. >>>> >>>> Maybe look at adding the backgroundProcessorDelay attribute >>>> to the context? I don't know what would happen if the context >>>> got a string of reload requests within the delay interval. >>>> Would it queue them up one after the other, or would it just >>>> execute one? > > I think it's more important to see what file(s) Tomcat thinks > is(are) being updated. I wonder if it's the same file, or if > there's some weird timestamp issue happening. Perhaps there is even > some kind of edge case where a resource's last-modified date isn't > being updated properly. > > In most cases, Tomcat reloads my application a single time, as > expected. These reload-storms are fairly rare. > > -chris >> >> --------------------------------------------------------------------- >> >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> >> > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJTPs0fAAoJEBzwKT+lPKRYYBcQAI+xdEUBXVdylZIWxcJoY097 v9xgkd6Zud4QoupVhUVrdP7YCjjhFTtS0y9MCOMxLdSyQQFjKS+yFfjE8Xtyah7x tnUwf200qnUBVCaBgDY+lfmrx+AuSNogce81qqpQkYUGiVJT+rW0MYnMQfgeCPmg RchOe6zo5664bvU3EBRnoPe3UYZcIwVjK9vXS8dy9vMygZJ6zGN+XIs0PTlFE5h/ HA8Ik6UJoK6yvyHqOfY2wC0xXeKMUsbNAcsIBcuC4H7h/NbDQCoH0YV5vvkpOBhF vhMLvCtzOgTEK0oLDBsbuiAXF6g1XzQk+Uxoy0a1Kx5U4TbswFIK2hHtAtmQWC1y v8RvvvNYNRO3tzI3Zo/CFO3+4Ha5EHmj96HLJI2itIOefD+By+y+gqqUR35lGdNT 7NY4zd0fIYQqkqkBajpDPJX6eGShEHX0obF5dR/46uaehl+Vy9tfUq+fZEILP2Cm XZdi6SD0epzpX+J6/7usQErU4QIxAnmmxwHu+ShhvRMF0+5ZsWCkt5RvZxMNuiMQ g9aDC1T+RACVOTCe80WDKgOnWmeGhbCycClb2ITSt3ceh/tb+JOShJoIgNl8r1MN Hq1h/aNxYLJRsKt3c4P+PXui4bgh8JVe3gtAuilo1s+0Z7gBgaAmwIVtG3gDA/hH ERlGxSO1fiY+0/p3Lf+f =3VV0 -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
