Hey Niall,
Thanks for your script, but I'm getting these errors:
./capture-diagnostics.sh RemoteCacheServerFactory
Capturing diagnostics for Java process "RemoteCacheServerFactory" (pid
2007)...
2007: Unable to open socket file: target process not responding or
HotSpot VM not loaded
The -F option can be used when the target process is not responding
2007: Unable to open socket file: target process not responding or
HotSpot VM not loaded
The -F option can be used when the target process is not responding
Saved diagnostics for "RemoteCacheServerFactory" to
"RemoteCacheServerFactory-diagnostics.txt"
There must be something I'm missing when I'm running the cache server. I
noticed it uses the 'server' VM by default, maybe these debug commands
are only good for the client VM?
Thanks!
-Josh
Niall Gallagher wrote:
Hi Josh,
Can you modify your cron job to capture diagnostics before it restarts
the cache server?
Then you can post the diagnostics next time it happens. The script below
will capture diagnostics for you. We use something like this in-house
for troubleshooting (not specifically for JCS).
You'll first have to run the JDK 'jps' command from either root, or the
user account which runs your cache server instance. This gives you the
"name" of your cache server JVM process, which you need to supply to the
diagnostics script as command-line parameter. The script uses the name
to attach to the relevant JVM process.
I don't know what might be causing the problem for you. It could be a
bug in JCS, or it could be a memory issue. The diagnostics will help
identify the problem.
Save this as "capture-diagnostics.sh"...
-------
#!/bin/sh
# Saves the stack traces and class memory usage information for a
# Java process running on the machine to a diagnostics file.
#
# This script expects the name of the relevant Java process to be
# specified as a parameter. The name specified should match a Java
# process name as listed by running the JDK 'jps' command.
#
# Usage: sh capture-diagnostics.sh <name of process>
APP_NAME="$1"
JDK_LOCATION="/usr/java/default"
DUMP_FILE="$APP_NAME-diagnostics.txt"
APP_PID="`$JDK_LOCATION/bin/jps|grep $APP_NAME 2> /dev/null|cut -d\
-f1`"
if [ "$APP_PID" = "" ]; then
echo "ERROR: Can't determine pid of Java process name specified
\"$APP_NAME\""
echo "Usage: sh capture-diagnostics.sh <name of process as listed by jps
command>"
exit 20
fi
echo "Capturing diagnostics for Java process \"$APP_NAME\" (pid
$APP_PID)..."
echo -e "Diagnostics for Java process \"$APP_NAME\" (pid $APP_PID) as at
`date`:-" >> $DUMP_FILE
echo -e "\nTop 30 memory-consuming classes:-" >> $DUMP_FILE
$JDK_LOCATION/bin/jmap -histo:live $APP_PID |head -n33 >> $DUMP_FILE
echo -e "\nThread stack traces:-" >> $DUMP_FILE
$JDK_LOCATION/bin/jstack $APP_PID >> $DUMP_FILE
echo -e "\n" >> $DUMP_FILE
echo "Saved diagnostics for \"$APP_NAME\" to \"$DUMP_FILE\""
-------
On Wed, 2008-04-09 at 10:11 -0400, Joshua Szmajda wrote:
Hey all,
I've got a JCS remote cache server running on a machine and every now
and then it will spiral out of control and lock the machine. I have no
idea yet what's causing this, I've just put some extra measures in place
to capture the logs from when it happens. My solution at this point is a
cron job that checks now and then for excessive cpu usage and restarts
the cache server. I'd like to be able to not worry about it, though :).
Any suggestions?
Thanks!
-Josh
P.S. it's running on ubuntu-server (kernel 2.6.22-14-server).
I have up to 16 remote listeners connecting to any given region.
(probably 20 application instances in all).
Puts grow at a rate of about 400 per second.
I pass these options to java: "-Xms128m -Xmx2000m"
And here's my simple remote.cache.ccf:
registry.host=localhost
registry.port=10021
remote.cluster.LocalClusterConsistency=true
remote.cluster.AllowClusterGet=true
jcs.default=DC
jcs.default.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
jcs.default.cacheattributes.MaxObjects=10000
jcs.default.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache
jcs.default.cacheattributes.UseMemoryShrinker=true
jcs.default.cacheattributes.MaxMemoryIdleTimeSeconds=3600
jcs.default.cacheattributes.ShrinkerIntervalSeconds=60
jcs.default.elementattributes=org.apache.jcs.engine.ElementAttributes
jcs.default.elementattributes.IsEternal=false
jcs.default.elementattributes.MaxLifeSeconds=86400
jcs.default.elementattributes.IdleTime=7200
jcs.default.elementattributes.IsSpool=true
jcs.default.elementattributes.IsRemote=true
jcs.default.elementattributes.IsLateral=true
jcs.auxiliary.DC=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheFactory
jcs.auxiliary.DC.attributes=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheAttributes
jcs.auxiliary.DC.attributes.DiskPath=/var/tmp/jcsServer
jcs.auxiliary.DC.attributes.MaxPurgatorySize=10000
jcs.auxiliary.DC.attributes.MaxKeySize=100000
jcs.auxiliary.DC.attributes.OptimizeAtRemoveCount=300000
jcs.auxiliary.DC.attributes.OptimizeOnShutdown=true
jcs.auxiliary.DC.attributes.MaxRecycleBinSize=7500
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
