On Mar 27, 2008, at 8:38 AM, Andrew Fritz wrote:
> Well, aside from preventing the crash/deadlock that occurred this
> morning, is there anything specific I should do to prep our cluster
> "prime time" use?
> We've had relatively light usage of our 2 server cluster for the last
> few months with very few problems and clustering seems to work well
> (with the exception of this morning). I've upped the -Xmx param on the
> servers to 75% of the physical memory (some time ago). I've configured
> our database cluster to allow 750 connections and upped Hibernate to
> 150 (for the moment) connections.
> Otherwise, we are starting with a stock config file.
> Any other suggestion?
The default <dependency-check-interval> should be raised to a minimum
of 60s or even larger.
If you haven't already profiled your application, remember that
Resin's /resin-admin has a profile tab. It's surprisingly
lightweight, so it's possible to use even on a production machine.
It's always a good idea to be aware of where your application is
spending its time.
Also, take a few thread dumps using the /resin-admin, so you get an
idea of the baseline behavior. If something does go wrong, you'll
want to be able to distinguish the normal behavior from the unusual.
(It'll also get you in the habit of taking thread dumps, for a freeze
or cpu spike. People always forget about thread dumps.)
On Linux (and other Unix), check your file descriptor max with uname -
a. The defaults are surprisingly low. (We're addressing this in
Remember that threads need virtual memory, too. That's an issue for
32-bit systems. 1024 threads x 1m stack size = 1G memory.
Become familiar with the jconsole view (or some other jmx admin).
> Any internal monitoring I can do on the servers themselves to
> how much of their resource pool is used up?
The /resin-admin status page has a basic overview. All the
administration is based on JMX, so you can either add some new php
pages to customize your view (it's pretty straightforward), or use
jconsole, or write your own Java-based JMX display (you can use
@javax.webbeans.In MBeanServer _server; to get the server), or use
some other third-party admin tool.
You'll want to set up administration and be familiar with normal,
baseline behavior before anything goes wrong.
Other people on the list probably have better suggestions. These are
mostly based on things people forget to do when they send in bug
reports, so it might not be representative of real-life administration.
> resin-interest mailing list
resin-interest mailing list