by default cassandra will use mmap data access on 64 bit environments. meaning, we don't _allocate_ anything off heap, but mmapped data counts towards resident memory as reported by top. the OS manages paging it in on demand.
On Fri, Feb 4, 2011 at 5:06 AM, Nick Telford <nick.telf...@gmail.com> wrote: > Hi guys, > > Users frequently ask questions regarding memory tuning and heap usage; one > of the more common being: "why does top report resident memory usage larger > than -Xmx?". > > I'd like to get some clarification on the usage patterns in Cassandra so I > write something up in the Wiki and generally answer questions better. > > I understand that the -Xm(s|x) flags only affect the JVM managed heap, and > that memory is also allocated for the stack (per-thread), program code and > native heap. Of these, I can only see the native heap being a major > contributor to memory usage. Would I be right in saying that this is due to > Cassandra's liberal use of NIO direct ByteBuffers? I couldn't find a whole > lot of information on JVM memory usage, but what I did find[1] suggests that > these are allocated outside of the native heap (with only a pointer object > in the JVM heap). > > If that's correct, what specifically are direct ByteBuffers used for and > when were they introduced? I seem to recall a lot of porting to ByteBuffers > in 0.7, yet our production 0.6 cluster exhibits a high non-JVM heap usage > that indicates otherwise. > > If I'm way off the mark, please could you guys point me in the right > direction. > > > [1]: http://www.ibm.com/developerworks/java/library/j-nativememory-linux/ > > Cheers, > > Nick Telford > -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
