Right. As a rule of thumb you should only use one of {key cache, row cache} at a time on a given CF.
On Tue, Mar 16, 2010 at 3:17 PM, B. Todd Burruss <bburr...@real.com> wrote: > i think i better make sure i understand how the row/key cache works. i > currently have both set to 10%. so if cassandra needs to read data from an > sstable that has 100 million rows, it will cache 10,000,000 rows of data > from that sstable? so if my row is ~4k, then we're looking at ~40gb used by > cache? > > Todd Burruss wrote: >> >> the row/key cache is set to 10%, but memory usage stays good until an >> anticompaction, hinted handoff, etc starts. (of course maybe i simply don't >> pay attention to memory until something bad happens) >> >> doesn't sound like anyone else is having trouble, so i'll keep review my >> settings for cache, keep testing and try to get something more concrete. >> >> Jonathan Ellis wrote: >> >>> >>> it's almost certainly GC storming due to memory pressure. (matching >>> the thread dump / the threads using the CPU in top will confirm this.) >>> reducing your cache sizes might be the best option since you already >>> have a 44GB heap. >>> >>> On Tue, Mar 16, 2010 at 12:17 PM, B. Todd Burruss <bburr...@real.com> >>> wrote: >>> >>>> >>>> thx, i'll try that next time, already restarted node .. but i will say >>>> the >>>> exact thing happened on another node as well. >>>> >>>> Jonathan Ellis wrote: >>>> >>>>> >>>>> You can still get a thread list w/ jstack, though. >>>>> >>>>> On Tue, Mar 16, 2010 at 11:46 AM, Gary Dusbabek <gdusba...@gmail.com> >>>>> wrote: >>>>> >>>>> >>>>>> >>>>>> On Tue, Mar 16, 2010 at 11:39, B. Todd Burruss <bburr...@real.com> >>>>>> wrote: >>>>>> >>>>>> >>>>>>> >>>>>>> any other ideas on how to troubleshoot? i have tried kill -3 >>>>>>> <java_pid> >>>>>>> in >>>>>>> the past but don't know where cassandra writes the console out. i'll >>>>>>> look >>>>>>> at scripts. >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> I have a sneaking suspicion that unless you're running with '-f' that >>>>>> the thread dump goes into the ether. The only logical place to look >>>>>> would be the log, which you've probably already done. >>>>>> >>>>>> Gary. >>>>>> >>>>>> >>>>>> >