I think we have seen similar symptoms, which turned out to be (unless my
memory deceives me) related to graphics. Resin was started via an ssh
session with X11 forwarding. After the app was up the ssh connection was
closed. At some point the app would want to generate some image or
something, and the JVM proceeds to delegate to X11, which now is utterly
unavailable. That in turned caused the JVM to abort abruptly.
That may be what happened, but don't quote me on that.
On Fri, Feb 11, 2011 at 11:40, Scott Ferguson <f...@caucho.com> wrote:
> Shane Cruz wrote:
> > We are running Resin Pro 3.0.25 on RHEL 5.5 and using 64-bit Sun JDK
> > 1.6.0_05. Recently, we have started seeing several incidents where
> > the Resin JVM seems to just randomly get restarted. There is nothing
> > in the logs to indicate that the JVM was shutdown cleanly or a restart
> > was attempted, the log files just go from displaying regular log lines
> > to displaying the following:
> The logging for 4.0 is much more informative. With 3.0 it's a bit trickier.
> > [11:24:18.095] com.caucho.log.EnvironmentLogger.log Server[myserver]
> > starting
> > Things that have already been checked:
> > 1. There doesn’t appear to be a JVM crash as no HotSpot Error log
> > files are created as they usually would be.
> > 2. There are no signs in the sudo logs that anyone is manually
> > restarting the JVM.
> > 3. There are no signs in the logs that Resin is restarting itself even
> > though we have a “min-free-memory” setting of 1M. With higher values
> > of that setting we have seen the JVM get restarted due to low memory,
> > but I am pretty sure logging always indicated that the JVM was
> > restarting when this happened before.
> > 4. We are not using the resin “ping” check that might restart the JVM
> > if it is unresponsive.
> > 5. Kernel logging is enabled and it doesn't look like the kernel
> > is killing it for any reason
> > It almost seems as if the JVM is just getting a kill -9 and then the
> > wrapper script is starting it back up. What is the best way to track
> > down what might be killing the JVM? We are in the process of testing
> > an upgrade to a newer version of the JDK, but I am not very confident
> > that will fix the problem. I am going to try to turn on full Resin
> > debug logging, but I thought I would reach out in case anyone else had
> > an idea of how to track this down. Is there a way to wrap the Linux
> > kill command to find out if that is being run? Any other suggestions
> > on where to look?
> Since a phantom kill is pretty unlikely, I wouldn't spend too much time
> on that theory.
> Since you're not getting a hs_* error, the most likely would be either
> something calling System.exit or System.halt, possibly Resin itself for
> something like running out of threads or memory (although, as you
> pointed out, that should be logged.)
> Other than that, the restart should only happen if the config files
> change (theoretically something like NFS or 'touch' could trigger that,
> but I assume that's not happening.)
> -- Scott
> > ------------------------------------------------------------------------
> > _______________________________________________
> > resin-interest mailing list
> > email@example.com
> > http://maillist.caucho.com/mailman/listinfo/resin-interest
> resin-interest mailing list
resin-interest mailing list