Hi Paulo, don't you think it might be better to keep applying the migration procedure whatever the version ? Anticompaction is pretty expensive on big SSTables and if the cluster has a lot of data, the first run might be very very long if the nodes are dense, and especially with a high number of vnodes. We've seen this on clusters that had just upgraded from 2.1 to 3.0, where the first incremental repair was taking a ridiculous amount of time as there were loads of anticompaction running.
Indeed, if you run an inc repair on all ranges on a node, it can skip anticompation by just marking SSTables as being repaired (which is fast), but the rest of the nodes will still have to perform anticompaction as they won't share all of its token ranges. Right ? Cheers, Alex Le lun. 12 sept. 2016 à 13:56, Paulo Motta <pauloricard...@gmail.com> a écrit : > > Can you clarify me please if what you said here applies for the version > 2.1.14. > > yes > > 2016-09-06 5:50 GMT-03:00 Jean Carlo <jean.jeancar...@gmail.com>: > >> Hi Paulo >> >> Can you clarify me please if what you said here >> >> 1. Migration procedure is no longer necessary after CASSANDRA-8004, and >> since you never ran repair before this would not make any difference >> anyway, so just run repair and by default (CASSANDRA-7250) this will >> already be incremental. >> >> applies for the version 2.1.14. I ask because I see that the jira >> CASSANDRA-8004 is resolved for the version 2.1.2 and we are considering to >> migrate to repairs inc before go to the version 3.0.x >> >> Thhx :) >> >> >> Saludos >> >> Jean Carlo >> >> "The best way to predict the future is to invent it" Alan Kay >> >> On Fri, Aug 26, 2016 at 9:04 PM, Stefano Ortolani <ostef...@gmail.com> >> wrote: >> >>> An extract of this conversation should definitely be posted somewhere. >>> Read a lot but never learnt all these bits... >>> >>> On Fri, Aug 26, 2016 at 2:53 PM, Paulo Motta <pauloricard...@gmail.com> >>> wrote: >>> >>>> > I must admit that I fail to understand currently how running repair >>>> with -pr could leave unrepaired data though, even when ran on all nodes in >>>> all DCs, and how that could be specific to incremental repair (and would >>>> appreciate if someone shared the explanation). >>>> >>>> Anti-compaction, which marks tables as repaired, is disabled for >>>> partial range repairs (which includes partitioner-range repair) to avoid >>>> the extra I/O cost of needing to run anti-compaction multiple times in a >>>> node to repair it completely. For example, there is an optimization which >>>> skips anti-compaction for sstables fully contained in the repaired range >>>> (only the repairedAt field is mutated), which is leveraged by full range >>>> repair, which would not work in many cases for partial range repairs, >>>> yielding higher I/O. >>>> >>>> 2016-08-26 10:17 GMT-03:00 Stefano Ortolani <ostef...@gmail.com>: >>>> >>>>> I see. Didn't think about it that way. Thanks for clarifying! >>>>> >>>>> >>>>> On Fri, Aug 26, 2016 at 2:14 PM, Paulo Motta <pauloricard...@gmail.com >>>>> > wrote: >>>>> >>>>>> > What is the underlying reason? >>>>>> >>>>>> Basically to minimize the amount of anti-compaction needed, since >>>>>> with RF=3 you'd need to perform anti-compaction 3 times in a particular >>>>>> node to get it fully repaired, while without it you can just repair the >>>>>> full node's range in one run. Assuming you run repair frequent enough >>>>>> this >>>>>> will not be a big deal, since you will skip already repaired data in the >>>>>> next round so you will not have the problem of re-doing work as in >>>>>> non-inc >>>>>> non-pr repair. >>>>>> >>>>>> 2016-08-26 7:57 GMT-03:00 Stefano Ortolani <ostef...@gmail.com>: >>>>>> >>>>>>> Hi Paulo, could you elaborate on 2? >>>>>>> I didn't know incremental repairs were not compatible with -pr >>>>>>> What is the underlying reason? >>>>>>> >>>>>>> Regards, >>>>>>> Stefano >>>>>>> >>>>>>> >>>>>>> On Fri, Aug 26, 2016 at 1:25 AM, Paulo Motta < >>>>>>> pauloricard...@gmail.com> wrote: >>>>>>> >>>>>>>> 1. Migration procedure is no longer necessary after CASSANDRA-8004, >>>>>>>> and since you never ran repair before this would not make any >>>>>>>> difference >>>>>>>> anyway, so just run repair and by default (CASSANDRA-7250) this will >>>>>>>> already be incremental. >>>>>>>> 2. Incremental repair is not supported with -pr, -local or -st/-et >>>>>>>> options, so you should run incremental repair in all nodes in all DCs >>>>>>>> sequentially (you should be aware that this will probably generate >>>>>>>> inter-DC >>>>>>>> traffic), no need to disable autocompaction or stopping nodes. >>>>>>>> >>>>>>>> 2016-08-25 18:27 GMT-03:00 Aleksandr Ivanov <ale...@gmail.com>: >>>>>>>> >>>>>>>>> I’m new in Cassandra and trying to figure out how to _start_ using >>>>>>>>> incremental repairs. I have seen article about “Migrating to >>>>>>>>> incremental >>>>>>>>> repairs” but since I didn’t use repairs before at all and I use >>>>>>>>> Cassandra >>>>>>>>> version v3.0.8, then maybe not all steps are needed which are >>>>>>>>> mentioned in >>>>>>>>> Datastax article. >>>>>>>>> Should I start with full repair or I can start with executing >>>>>>>>> “nodetool repair -pr my_keyspace” on all nodes without autocompaction >>>>>>>>> disabling and node stopping? >>>>>>>>> >>>>>>>>> I have 6 datacenters with 6 nodes in each DC. Is it enough to run >>>>>>>>> “nodetool repair -pr my_keyspace” in one DC only or it should be >>>>>>>>> executed >>>>>>>>> on all nodes in _all_ DCs? >>>>>>>>> >>>>>>>>> I have tried to perform “nodetool repair -pr my_keyspace” on all >>>>>>>>> nodes in all datacenters sequentially but I still can see non repaired >>>>>>>>> SSTables for my_keyspace (Repaired at: 0). Is it expected behavior >>>>>>>>> if >>>>>>>>> during repair data in my_keyspace wasn’t modified (no writes, no >>>>>>>>> reads)? >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
