Re: memory estimate for each key in the key cache

2011-12-20 Thread Kent Tong
 It is not telling you to multiply your key size by 10-12, it is telling you to multiply the output of the nodetool cfstats reported key cache size by 10-12. The  key cache size reported is actually the number of keys in the key cache. So, it is the same thing as suggesting each key takes

Re: memory estimate for each key in the key cache

2011-12-20 Thread aaron morton
If you get the sizing wrong there area couple of emergency pressure valves in the config… https://github.com/apache/cassandra/blob/trunk/conf/cassandra.yaml#L113 Watch for log messages with Cassandra is now reducing cache sizes to free up memory. in them. Cheers - Aaron

Re: memory estimate for each key in the key cache

2011-12-19 Thread Jeremiah Jordan
It is not telling you to multiply your key size by 10-12, it is telling you to multiply the output of the nodetool cfstats reported key cache size by 10-12. -Jeremiah On Dec 18, 2011, at 6:37 PM, Guy Incognito wrote: to be blunt, this doesn't sound right to me, unless it's doing something

Re: memory estimate for each key in the key cache

2011-12-18 Thread Guy Incognito
to be blunt, this doesn't sound right to me, unless it's doing something rather more clever to manage the memory. i mocked up a simple class containing a byte[], ByteBuffer and long, and the shallow size alone is 32 bytes. deep size with a byte[16], 1-byte bytebuffer and long is 132. this

Re: memory estimate for each key in the key cache

2011-12-16 Thread Brandon Williams
On Fri, Dec 16, 2011 at 8:52 PM, Kent Tong freemant2...@yahoo.com wrote: Hi, From the source code I can see that for each key, the hash (token), the key itself (ByteBuffer) and the position (long. offset in the sstable) are stored into the key cache. The hash is an MD5 hash, so it is 16

Re: memory estimate for each key in the key cache

2011-12-16 Thread Dave Brosius
On 12/16/2011 10:13 PM, Brandon Williams wrote: On Fri, Dec 16, 2011 at 8:52 PM, Kent Tongfreemant2...@yahoo.com wrote: Hi, From the source code I can see that for each key, the hash (token), the key itself (ByteBuffer) and the position (long. offset in the sstable) are stored into the key

Re: memory estimate for each key in the key cache

2011-12-16 Thread Brandon Williams
On Fri, Dec 16, 2011 at 9:31 PM, Dave Brosius dbros...@mebigfatguy.com wrote: Wow, Java is a lot better than I thought if it can perform that kind of magic.  I'm guessing the wiki information is just old and out of date. It's probably more like 60 + sizeof(key) With jamm and MAT it's fairly