Re: very slow repair

2019-06-13 Thread Oleksandr Shulgin
On Thu, Jun 13, 2019 at 2:09 PM Léo FERLIN SUTTON wrote: > Last, but not least: are you using the default number of vnodes, 256? The >> overhead of large number of vnodes (times the number of nodes), can be >> quite significant. We've seen major improvements in repair runtime after >>

Re: very slow repair

2019-06-13 Thread Léo FERLIN SUTTON
> > Last, but not least: are you using the default number of vnodes, 256? The > overhead of large number of vnodes (times the number of nodes), can be > quite significant. We've seen major improvements in repair runtime after > switching from 256 to 16 vnodes on Cassandra version 3.0. Is there

Re: very slow repair

2019-06-13 Thread Oleksandr Shulgin
On Thu, Jun 13, 2019 at 10:36 AM R. T. wrote: > > Well, actually by running cfstats I can see that the totaldiskspaceused is > about ~ 1.2 TB per node in the DC1 and ~ 1 TB per node in DC2. DC2 was off > for a while thats why there is a difference in space. > > I am using Cassandra 3.0.6 and >

Re: very slow repair

2019-06-13 Thread R. T.
Hi, Thank you for your reply, Well, actually by running cfstats I can see that the totaldiskspaceused is about ~ 1.2 TB per node in the DC1 and ~ 1 TB per node in DC2. DC2 was off for a while thats why there is a difference in space. I am using Cassandra 3.0.6 and my

Re: very slow repair

2019-06-12 Thread Laxmikant Upadhyay
Few queries: 1. What is the cassandra version ? 2. is the size of table 4TB per node ? 3. What is the value of compaction_throughput_mb_per_sec and stream_throughput_outbound_megabits_per_sec ? On Thu, Jun 13, 2019 at 5:06 AM R. T. wrote: > Hi, > > I am trying to run a repair for first time a

very slow repair

2019-06-12 Thread R. T.
Hi, I am trying to run a repair for first time a specific column family in specific keyspace and it seems that is going super slow. I have 6 nodes cluster with 2 Datacenters (RF 2) and the repair is a non incremental, DC parallel one. This column family is around 4 TB and it is written

Re: Slow repair

2017-03-17 Thread Gábor Auth
Hi, On Wed, Mar 15, 2017 at 11:35 AM Ben Slater wrote: > When you say you’re running repair to “rebalance” do you mean to populate > the new DC? If so, the normal/correct procedure is to use nodetool rebuild > rather than repair. > Oh, thank you! :) Bye, Gábor Auth

Re: Slow repair

2017-03-16 Thread siddharth verma
Hi, We did a similar thing when a new DC was added and had to populate it according to altered replication of keyspace. For repair we used Tickler approach rather than actual nodetool repair. (using the blocking read repair feature in cassandra) You can see 1. Ticker by ckalantzis :

Re: Slow repair

2017-03-15 Thread Ben Slater
When you say you’re running repair to “rebalance” do you mean to populate the new DC? If so, the normal/correct procedure is to use nodetool rebuild rather than repair. See https://docs.datastax.com/en/cassandra/2.1/cassandra/operations/ops_add_dc_to_cluster_t.html for the full details. Cheers

Slow repair

2017-03-15 Thread Gábor Auth
Hi, We are working with a two DCs Cassandra cluster (EU and US), so that the distance is over 160 ms between them. I've added a new DC to this cluster, modified the keyspace's replication factor and trying to rebalance it with repair but the repair is very slow (over 10-15 minutes per node per