Re: non incremental repairs with cassandra 2.2+

2016-10-20 Thread kurt Greaves
probably because i was looking the wrong version of the codebase :p

Re: non incremental repairs with cassandra 2.2+

2016-10-20 Thread kurt Greaves
Welp, that's good but wasn't apparent in the codebase :S. Kurt Greaves k...@instaclustr.com www.instaclustr.com On 20 October 2016 at 05:02, Alexander Dejanovski wrote: > Hi Kurt, > > we're not actually. > Reaper performs full repair by subrange but does incremental

Re: non incremental repairs with cassandra 2.2+

2016-10-19 Thread Alexander Dejanovski
Hi Kurt, we're not actually. Reaper performs full repair by subrange but does incremental repair on all ranges at once, node by node. Subrange is incompatible with incremental repair anyway. Cheers, On Thu, Oct 20, 2016 at 5:24 AM kurt Greaves wrote: > > On 19 October

Re: non incremental repairs with cassandra 2.2+

2016-10-19 Thread kurt Greaves
On 19 October 2016 at 17:13, Alexander Dejanovski wrote: > There aren't that many tools I know to orchestrate repairs and we maintain > a fork of Reaper, that was made by Spotify, and handles incremental repair > : https://github.com/thelastpickle/cassandra-reaper Looks

Re: non incremental repairs with cassandra 2.2+

2016-10-19 Thread Kant Kodali
Sorry I shouldn't have said adding a node. Sometimes data seems to be corrupted or inconsistent in which case would like to run a repair. Sent from my iPhone > On Oct 19, 2016, at 10:10 AM, Sean Bridges > wrote: > > Thanks, we will try that. > > Sean > >> On

Re: non incremental repairs with cassandra 2.2+

2016-10-19 Thread Alexander Dejanovski
There aren't that many tools I know to orchestrate repairs and we maintain a fork of Reaper, that was made by Spotify, and handles incremental repair : https://github.com/thelastpickle/cassandra-reaper We just added Cassandra as storage back end (only postgres currently) in one of the branches,

Re: non incremental repairs with cassandra 2.2+

2016-10-19 Thread Sean Bridges
Thanks, we will try that. Sean On 16-10-19 09:34 AM, Alexander Dejanovski wrote: Hi Sean, you should be able to do that by running subrange repairs, which is the only type of repair that wouldn't trigger anticompaction AFAIK. Beware that now you will have sstables marked as repaired and

Re: non incremental repairs with cassandra 2.2+

2016-10-19 Thread Alexander Dejanovski
Can you explain why you would want to run repair for new nodes? Aren't you talking about bootstrap, which is not related to repair actually? Le mer. 19 oct. 2016 18:57, Kant Kodali a écrit : > Thanks! How do I do an incremental repair when I add a new node? > > Sent from my

Re: non incremental repairs with cassandra 2.2+

2016-10-19 Thread Kant Kodali
Also any suggestions on a tool to orchestrate the incremental repair? Like say most commonly used Sent from my iPhone > On Oct 19, 2016, at 9:54 AM, Alexander Dejanovski > wrote: > > Hi Kant, > > subrange is a form of full repair, so it will just split the repair

Re: non incremental repairs with cassandra 2.2+

2016-10-19 Thread Kant Kodali
Thanks! How do I do an incremental repair when I add a new node? Sent from my iPhone > On Oct 19, 2016, at 9:54 AM, Alexander Dejanovski > wrote: > > Hi Kant, > > subrange is a form of full repair, so it will just split the repair process > in smaller yet sequential

Re: non incremental repairs with cassandra 2.2+

2016-10-19 Thread Alexander Dejanovski
Hi Kant, subrange is a form of full repair, so it will just split the repair process in smaller yet sequential pieces of work (repair is started giving a start and end token). Overall, you should not expect improvements other than having less overstreaming and better chances of success if your

Re: non incremental repairs with cassandra 2.2+

2016-10-19 Thread Kant Kodali
Another question on a same note would be what would be the fastest way to do repairs of size 10TB cluster ? Full repairs are taking days. So among repair parallel or repair sub range which is faster in the case of say adding a new node to the cluster? Sent from my iPhone > On Oct 19, 2016, at

Re: non incremental repairs with cassandra 2.2+

2016-10-19 Thread Alexander Dejanovski
Hi Sean, you should be able to do that by running subrange repairs, which is the only type of repair that wouldn't trigger anticompaction AFAIK. Beware that now you will have sstables marked as repaired and others marked as unrepaired, which will never be compacted together. You might want to

non incremental repairs with cassandra 2.2+

2016-10-19 Thread Sean Bridges
Hey, We are upgrading from cassandra 2.1 to cassandra 2.2. With cassandra 2.1 we would periodically repair all nodes, using the -pr flag. With cassandra 2.2, the same repair takes a very long time, as cassandra does an anti compaction after the repair. This anti compaction causes most