Thanks for the response. With continuous runs I am observing my off heap memory increasing and *memory usage remains high even if no traffic is running*. Sometimes a node is hung with following error:
ERROR [ReadStage-5] 2020-03-12 20:47:46,406 JVMStabilityInspector.java:74 - OutOfMemory error letting the JVM handle the error: java.lang.OutOfMemoryError: Direct buffer memory at java.nio.Bits.reserveMemory(Bits.java:694) ~[na:1.8.0_221] at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) ~[na:1.8.0_221] at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311) ~[na:1.8.0_221] at org.apache.cassandra.utils.memory.BufferPool.allocate(BufferPool.java:110) ~[apache-cassandra-3.11.2.jar:3.11.2] at org.apache.cassandra.utils.memory.BufferPool.access$1000(BufferPool.java:46) As per my understanding, this error comes when memory reservation fails for Direct Byte buffer.I want to understand following things 1. Why Cassandra memory usage remains high even if no traffic is run? 2. In which case I will get above error ( *java.lang.OutOfMemoryError: Direct buffer memory*). How to avoid those cases? Regards Manish On Tue, Mar 24, 2020 at 7:50 AM Erick Ramirez <erick.rami...@datastax.com> wrote: > It's more indicative of your nodes not able to service requests because > they are busy or overloaded. In my experience when you see the number > climbing because of peak load, the latency tends to show issues too. Other > symptoms of the load issue are higher-than-normal pending reads and writes. > The high memory utilisation is also a side-effect of the high traffic load. > At its worst, you'd be seeing read/write timeouts or connection failures on > the client side. Cheers! > >>
