Re: Cassandra 3.0.14 transport completely blocked

2022-03-23 Thread Jaydeep Chovatia
Thank you all. I will try different options and will let you know which one worked for my case. On Wed, Mar 23, 2022 at 3:25 AM Bowen Song wrote: > I remember we had the same issue back in the Cassandra 2.x days, and > restarting the affected node only makes the issue go away temporarily. The >

Re: Cassandra 3.0.14 transport completely blocked

2022-03-23 Thread Bowen Song
I remember we had the same issue back in the Cassandra 2.x days, and restarting the affected node only makes the issue go away temporarily. The issue we had was "fixed" by adding "-Dcassandra.max_queued_native_transport_requests=4096" to the JVM options. I dug that option out from our old

Re: Cassandra 3.0.14 transport completely blocked

2022-03-22 Thread Erick Ramirez
> > Thanks, Scott, for the prompt response! We will apply this patch and see > how it goes. > Also, in the near future, we will consider upgrading to 3.0.26 and > eventually to 4.0 > We would really discourage you from just upgrading to C* 3.0.21. There really is no logical reason for doing that.

Re: Cassandra 3.0.14 transport completely blocked

2022-03-22 Thread Jaydeep Chovatia
Thanks, Scott, for the prompt response! We will apply this patch and see how it goes. Also, in the near future, we will consider upgrading to 3.0.26 and eventually to 4.0 Thanks a lot! On Tue, Mar 22, 2022 at 9:45 PM C. Scott Andreas wrote: > Hi Jaydeep, thanks for reaching out. > > The most

Re: Cassandra 3.0.14 transport completely blocked

2022-03-22 Thread C. Scott Andreas
Hi Jaydeep, thanks for reaching out.The most notable deadlock identified and resolved in the last few years is https://issues.apache.org/jira/browse/CASSANDRA-15367: Memtable memory allocations may deadlock (fixed in Apache Cassandra 3.0.21).Mentioning for completeness - since the release of

Cassandra 3.0.14 transport completely blocked

2022-03-22 Thread Jaydeep Chovatia
Hi, I have been using Cassandra 3.0.14 in production for a long time. Recently I have found a bug in that, all of a sudden the transport thread-pool hangs. *Observation:* If I do *nodetool tpstats*, then it shows *"Native-Transport-Requests"* is blocking "Active" tasks. I stopped the complete