Re: High value of "All Time Blocked" for "Native-Transport-Requests"

2020-03-30 Thread manish khandelwal
To add on to my previous email, I think we get this error when actual off heap memory breaches "off heap memory limit" as mentioned in this JIRA . On Mon, Mar 30, 2020 at 3:10 PM manish khandelwal < manishkhandelwa...@gmail.com> wrote: > I

Re: High value of "All Time Blocked" for "Native-Transport-Requests"

2020-03-30 Thread manish khandelwal
I have already updated that property to mmap_index_only. And in pmap I could see only index files are loaded into memory not the SSTables. I feel this is some to do with direct memory buffer. There is a opened JIRA ticket https://issues.apache.org/jira/browse/CASSANDRA-10930 as well regarding

Re: High value of "All Time Blocked" for "Native-Transport-Requests"

2020-03-30 Thread Erick Ramirez
> > 1. Why Cassandra memory usage remains high even if no traffic is run? > 2. In which case I will get above error ( *java.lang.OutOfMemoryError: > Direct buffer memory*). How to avoid those cases? > I don't have enough information to know if it applies to your case but there's a good

Re: High value of "All Time Blocked" for "Native-Transport-Requests"

2020-03-30 Thread manish khandelwal
Thanks for the response. With continuous runs I am observing my off heap memory increasing and *memory usage remains high even if no traffic is running*. Sometimes a node is hung with following error: ERROR [ReadStage-5] 2020-03-12 20:47:46,406 JVMStabilityInspector.java:74 - OutOfMemory error

Re: High value of "All Time Blocked" for "Native-Transport-Requests"

2020-03-23 Thread Erick Ramirez
It's more indicative of your nodes not able to service requests because they are busy or overloaded. In my experience when you see the number climbing because of peak load, the latency tends to show issues too. Other symptoms of the load issue are higher-than-normal pending reads and writes. The

High value of "All Time Blocked" for "Native-Transport-Requests"

2020-03-23 Thread manish khandelwal
I am working with Cassandra 3.11.2. In nodetool tpstats output I am observing high number in "All Time Blocked" column for "Native-Transport-Requests" pool. I have two questions 1. Does a high number of "ALL Time Blocked" count impact client latency? 2. Does it increase off head memory overtime?