Hinted handoff throttled even after "nodetool sethintedhandoffthrottlekb 0"

2017-10-26 Thread Andrew Bialecki
We have a 96 node cluster running 3.11 with 256 vnodes each. We're running a rolling restart. As we restart nodes, we notice that each node takes a while to have all other nodes be marked as up and this corresponds to nodes that haven't finished playing hints. We looked at the hinted handoff

Re: server connection in authenticator

2017-10-26 Thread Justin Cameron
Hi Horia, Are you aware that Cassandra already supports two-way SSL certificate authentication? Take a look at the require_client_auth option under client_encryption_options in cassandra.yaml:

Materialized Views marked experimental

2017-10-26 Thread Blake Eggleston
Hi user@, Following a discussion on dev@, the materialized view feature is being retroactively classified as experimental, and not recommended for new production uses. The next patch releases of 3.0, 3.11, and 4.0 will include CASSANDRA-13959, which will log warnings when materialized views

Re: Getting DigestMismatchExceptions despite setting read repair chances to zero

2017-10-26 Thread Artur Siekielski
It was set to the default 99PERCENTILE, I changed it to NONE but the exceptions are still logged (for the same table). I'm assuming node restarts are not required for that ALTER. On 10/26/2017 05:13 PM, Jeff Jirsa wrote: Is speculative retry enabled?

Re: Getting DigestMismatchExceptions despite setting read repair chances to zero

2017-10-26 Thread Jeff Jirsa
Is speculative retry enabled? -- Jeff Jirsa > On Oct 26, 2017, at 3:19 AM, Artur Siekielski wrote: > > Hi, > > we have one table for which reads and writes are done with CL=ONE. The table > contains counters. We wanted to disable async read repair for the table (to > lessen

RE: how to identify the root cause of cassandra hang

2017-10-26 Thread Meg Mara
Hello, It could happen if your GC pauses are too long and/or too frequent. If your heap sizes are not large enough. When a long GC happens, Cassandra node effectively behaves like a dead node (unresponsive). Other nodes start collecting hints for it etc. Maybe you should look into your logs to

how to identify the root cause of cassandra hang

2017-10-26 Thread Peng Xiao
Hi, We have a cluster with 48 nodes configured with RACK,sometimes it's hang for even 2 minutes.the response time jump from 300ms to 15s. Could anyone please advise how to identified the root cause ? The following is from the system log INFO [Service Thread] 2017-10-26 21:45:46,796

Getting DigestMismatchExceptions despite setting read repair chances to zero

2017-10-26 Thread Artur Siekielski
Hi, we have one table for which reads and writes are done with CL=ONE. The table contains counters. We wanted to disable async read repair for the table (to lessen cluster load and to avoid DigestMismatchExceptions in debug.log). After altering the table with read_repair_chance=0,

Re: server connection in authenticator

2017-10-26 Thread Horia Mocioi
Thank you Jeff & Harika. Yes, I am aware of that mechanism. What we need to do is to add some extra validations on the certificate used for securing the connection.  So, in order to do this in our Authenticator, we need a way to grab the sslHandler which can be obtained from the