it's almost certainly GC storming due to memory pressure. (matching the thread dump / the threads using the CPU in top will confirm this.) reducing your cache sizes might be the best option since you already have a 44GB heap.
On Tue, Mar 16, 2010 at 12:17 PM, B. Todd Burruss <bburr...@real.com> wrote: > thx, i'll try that next time, already restarted node .. but i will say the > exact thing happened on another node as well. > > Jonathan Ellis wrote: >> >> You can still get a thread list w/ jstack, though. >> >> On Tue, Mar 16, 2010 at 11:46 AM, Gary Dusbabek <gdusba...@gmail.com> >> wrote: >> >>> >>> On Tue, Mar 16, 2010 at 11:39, B. Todd Burruss <bburr...@real.com> wrote: >>> >>>> >>>> any other ideas on how to troubleshoot? i have tried kill -3 <java_pid> >>>> in >>>> the past but don't know where cassandra writes the console out. i'll >>>> look >>>> at scripts. >>>> >>>> >>> >>> I have a sneaking suspicion that unless you're running with '-f' that >>> the thread dump goes into the ether. The only logical place to look >>> would be the log, which you've probably already done. >>> >>> Gary. >>> >>> >
