Re: too many warnings of Heap is full

2013-02-01 Thread Guillermo Barbero
What is the cardinality like on these indexes? Can you provide the schema creation for these two column families? This is the schema of the CFs: create column family CF_users with comparator = UTF8Type and column_metadata = [

Re: too many warnings of Heap is full

2013-01-31 Thread Alain RODRIGUEZ
@Bryan Other space is used but can be more easily controlled by tuning for - memtable - compaction - key cache - *row cache* Isn't row cache stored off-heap since a while ? 2013/1/31 Bryan Talbot btal...@aeriagames.com On Wed, Jan 30, 2013 at 2:44 PM, Guillermo Barbero

Re: too many warnings of Heap is full

2013-01-31 Thread Guillermo Barbero
Hi, replies to your answers: The CFs that seems to flush more often (users and messages) do not have multiget queries... There are 4 secondary indexes for users and 3 for messages. I've seen something odd here: the index of the CFs is also stored in a column AND is also indexed. There isn't any

Re: too many warnings of Heap is full

2013-01-31 Thread Nate McCall
There are 4 secondary indexes for users and 3 for messages. I've seen something odd here: the index of the CFs is also stored in a column AND is also indexed. There isn't any query by that secondary index. I should probably erase the index, shouldn't I? What is the cardinality like on these

Re: too many warnings of Heap is full

2013-01-30 Thread Bryan Talbot
My guess is that those one or two nodes with the gc pressure also have more rows in your big CF. More rows could be due to imbalanced distribution if your'e not using a random partitioner or from those nodes not yet removing deleted rows which other nodes may have done. JVM heap space is used

Re: too many warnings of Heap is full

2013-01-30 Thread Nate McCall
What's the output of nodetool cfstats for those 2 column families on cassNode2 and cassNode3? And what is the replication factor for this cluster? Per the previous reply, nodetool ring should show each of your nodes with ~16.7% of the data if well balanced. Also, the auto-detection for memory

Re: too many warnings of Heap is full

2013-01-30 Thread Nate McCall
Your latencies and distribution look fine. How big/what types of queries are you issuing? Are you issuing a lot of large multigets? Also, do either of these column families have secondary indexes? On Wed, Jan 30, 2013 at 2:59 PM, Guillermo Barbero guillermo.barb...@spotbros.com wrote: Iep,