The 2 largest users of offheap memory are bloom filters (a probabilistic data structure that lets cassandra skip some sstables on the read path) and compression metadata chunks (data structure that helps us track / use offsets in compressed files).
Bloom filters you can tune by changing the false positive ratio - higher tolerated false positive %, the less RAM required Offheap compression metadata you can tune by having fewer chunks (== larger chunks == if we have to go to disk, we'll have to read+decompress more data). On Mon, Jun 19, 2017 at 12:28 AM, Павел Сапежко <amelius0...@gmail.com> wrote: > Hi! We use cassandra inside kubernetes cluster. Now I try to find out how > cassandra uses off-heap memory. Currently we have 10 GB heap, and 21GB > memory limit(k8s limit) per cassandra node. So in attach you can find our > cassandra.yaml, k8s statefulset, jvm.options and results of execution some > tools like jcmd, output of /proc/<pid>/stats, jolokia responses and start > logs of cassandra node. > > So now I have several questions. > > 1. What does it mean "internal" memory in native_memory output? > 2. Why in detailed jcmd native_memory ouput I see the next lines? For > what cassandra uses so much off-heap memory(allocated as one big chunk)? > [0x00007f925a29fff3] Unsafe_AllocateMemory+0xc3 > [0x00007f924a3b4368] > (malloc=1790102KB #628) > 3. Why some java thread(as you can see in native memory output) use > 1028KB stack, although I used -Xss256K JVM option? > 4. Why after disabling mmap(disk_access_mode property in > cassandra.yaml) cassandra still use mmap as can be see in properly jolokia > response in attach? > 5. For what cassandra uses direct ByteBuffer pools and can I somehow > controlled it(set some limits)? > > Some environment details. > As I sad above, we use cassandra inside k8s cluster. As storage we use rbd > from ceph pool based on ssd disks. We run cassandra on Oracle JVM version > 8_131 inside docker, of cause. Cassandra version - latest LTS 3.0.13 > P.S. There is no answer in user channel > -- > > С уважением, > > Павел Сапежко > > skype: p.sapezhko > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org >
