Re: Unexplainable spikes of requests latency

2018-12-13 Thread Nitan Kainth
Latency and mismatch matches may not align but mismatch means data is not in sync AND can cause read latency. On Thu, Dec 13, 2018 at 2:34 AM Виталий Савкин wrote: > Good catch. We ran repairs few times but don't do it on a regular basis. > But I found no dependency between count of

Re: Unexplainable spikes of requests latency

2018-12-13 Thread Виталий Савкин
Good catch. We ran repairs few times but don't do it on a regular basis. But I found no dependency between count of DigestMismatchExceptions and latency spikes (see attached graphs for example). One important point I didn't mention in the original mail is that all requests (both reads and writes)

Re: Unexplainable spikes of requests latency

2018-12-12 Thread Nitan Kainth
DigestMismatchExceptions --> could be due to data out of sync.Are you running repairs? On Wed, Dec 12, 2018 at 11:39 AM Виталий Савкин wrote: > Hi everyone! > > Few times a day I see spikes of requests latencies on my cassandra > clients. Usually 99thPercentile is below 100ms but that times

Unexplainable spikes of requests latency

2018-12-12 Thread Виталий Савкин
Hi everyone! Few times a day I see spikes of requests latencies on my cassandra clients. Usually 99thPercentile is below 100ms but that times it grows above 1 second. Type of request doesn't matter: different services are affected and I found that three absolutely identical requests (to the same