Re: Bootstrapping node on Cassandra 3.7 causes cluster-wide performance issues

Thanks again guys, this has been a major blocker for us and I think we've made some major progress with your advice. We have gone ahead with Lerh's suggestion and the cluster is operating much more smoothly while the new node compacts. We read at quorum, so in the event that we don't make it

Re: Bootstrapping node on Cassandra 3.7 causes cluster-wide performance issues

> > > Kurt - We're on 3.7, and our approach was to try thorttling compaction > throughput as much as possible rather than the opposite. I had found some > resources that suggested unthrottling to let it get it over with, but > wasn't sure if this would really help in our situation since the I/O

Re: Bootstrapping node on Cassandra 3.7 causes cluster-wide performance issues

Hi Paul, Agh, I don't have any experience with sstableofflinerelevel. Maybe Kurt does, sorry. Also, if it wasn't obvious, to add back the node to the cluster once it is done would be the 3 commands, with enable substituted for disable. It feels like it will take some time to get through all the

Re: Bootstrapping node on Cassandra 3.7 causes cluster-wide performance issues

Unsubscribe On Mon, Sep 11, 2017 at 4:48 PM, Paul Pollack wrote: > Hi, > > We run 48 node cluster that stores counts in wide rows. Each node is using > roughly 1TB space on a 2TB EBS gp2 drive for data directory and > LeveledCompactionStrategy. We have been trying to

Re: Bootstrapping node on Cassandra 3.7 causes cluster-wide performance issues

Thanks for the responses Lerh and Kurt! Lerh - We had been considering those particular nodetool commands but were hesitant to perform them on a production node without either testing adequately in a dev environment or getting some feedback from someone who knew what they were doing (such as

Re: Bootstrapping node on Cassandra 3.7 causes cluster-wide performance issues

What version are you using? There are improvements to streaming with LCS in 2.2. Also, are you unthrottling compaction throughput while the node is bootstrapping? ​

Re: Bootstrapping node on Cassandra 3.7 causes cluster-wide performance issues

Hi Paul, The new node will certainly have a lot of compactions to deal with being LCS. Have you tried performing the following on the new node once it has joined? *nodetool disablebinary && nodetool disablethrift && nodetooldisablegossip* This will disconnect Cassandra from the cluster, but not

Bootstrapping node on Cassandra 3.7 causes cluster-wide performance issues

Hi, We run 48 node cluster that stores counts in wide rows. Each node is using roughly 1TB space on a 2TB EBS gp2 drive for data directory and LeveledCompactionStrategy. We have been trying to bootstrap new nodes that use a raid0 configuration over 2 1TB EBS drives to increase I/O throughput cap