Re: Cassandra seems slow when having many read operations

On 2017-07-13 07:49 (-0700), Felipe Esteves wrote: > Hi, > > I have a Cassandra 2.1 cluster running on AWS that receives high read > loads, jumping from 100k requests to 400k requests, for example. Then it > normalizes and later cames another high throughput. >

Re: Cassandra seems slow when having many read operations

On Sat, Jul 15, 2017 at 6:37 AM, Felipe Esteves < felipe.este...@b2wdigital.com> wrote: > > One point I've noticed, is that Opscenter show "OS: Disk Latency" max with > high values when the problem occurs, but it doesn't reflect in server > directly monitoring, in these tools the IO and latency

Re: Cassandra seems slow when having many read operations

Chunk size: For us it made a 20x difference in read io. But it depends a lot on the use case. Am 22.07.2017 08:32 schrieb "Fay Hou [Storage Service] ­" < fay...@coupang.com>: > Hey Felipe: > > When you say increased memory from 16GB to 24GB, I think you meant you > increased heap to 24GB. do you

Re: Cassandra seems slow when having many read operations

Hey Felipe: When you say increased memory from 16GB to 24GB, I think you meant you increased heap to 24GB. do you use cms or g1gc? did you change any other parameters? As for the chunk size, we found change 64kb to 16kb didn't make a difference in low key cache rate environment On Fri, Jul

Re: Cassandra seems slow when having many read operations

Apart from all that you can try to reduce the compression chunk size from the default 64kb to 16kb or even down to 4kb. This can help a lot if your read io on disk is very high and the page cache is not efficient. Am 21.07.2017 23:03 schrieb "Petrus Gomes" : > Thanks a lot to

Re: Cassandra seems slow when having many read operations

Thanks a lot to share the result. Boa Sorte. ;-) Take care. Petris Silva On Fri, Jul 21, 2017 at 12:19 PM, Felipe Esteves < felipe.este...@b2wdigital.com> wrote: > Hi, Petrus, > > Seems we've solved the problem, but it wasn't relationed to repair the > cluster or disk latency. > I've increased

Re: Cassandra seems slow when having many read operations

Hi, Petrus, Seems we've solved the problem, but it wasn't relationed to repair the cluster or disk latency. I've increased the memory available for Cassandra from 16GB to 24GB and the performance was much improved! The main symptom we've observed in Opscenter was a significantly decrease in total

Re: Cassandra seems slow when having many read operations

Hi Felipe, Yes, try it and let us know how it goes. Thanks, Petrus Silva. On Fri, Jul 14, 2017 at 11:37 AM, Felipe Esteves < felipe.este...@b2wdigital.com> wrote: > Hi Petrus, thanks for the feedback. > > I couldn't found the percent repaired in nodetool info, C* version is > 2.1.8, maybe it's

Re: Cassandra seems slow when having many read operations

Hi Petrus, thanks for the feedback. I couldn't found the percent repaired in nodetool info, C* version is 2.1.8, maybe it's something newer than that? I'm analyzing this thread about num_token. Compaction is "compaction_throughput_mb_per_sec: 16", I don't get pending compactions in Opscenter.

Re: Cassandra seems slow when having many read operations

How is your Percent Repaired when you run " nodetool info" ? Search for : "reduced num_token = improved performance ??" topic. The people were discussing that. How is your compaction is configured? Could you run the same process in command line to have a measurement? Thanks, Petrus Silva

Cassandra seems slow when having many read operations

Hi, I have a Cassandra 2.1 cluster running on AWS that receives high read loads, jumping from 100k requests to 400k requests, for example. Then it normalizes and later cames another high throughput. To the application, it appears that Cassandra is slow. However, cpu and disk use is ok in every