Re: Automating nodetool repair

2012-08-29 Thread aaron morton
Staggering the repairs also gives the DynamicSnitch a chance to route around nodes which maybe running slow. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 29/08/2012, at 11:19 AM, Omid Aladini omidalad...@gmail.com wrote: Secondly,

Re: Automating nodetool repair

2012-08-28 Thread Edward Capriolo
You can consider adding -pr. When iterating through all your hosts like this. -pr means primary range, and will do less duplicated work. On Mon, Aug 27, 2012 at 8:05 PM, Aaron Turner synfina...@gmail.com wrote: I use cron. On one box I just do: for n in node1 node2 node3 node4 ; do

Re: Automating nodetool repair

2012-08-28 Thread Aaron Turner
Funny you mention that... i just was hearing on #cassandra this morning that it repairs the replica set by default. I was thinking of repairing every 3rd node (RF=3), but running -pr seems cleaner. Do you know if this (repairing a replica vs node) was introduced in 1.0 or 1.1? On Tue, Aug 28,

Re: Automating nodetool repair

2012-08-28 Thread Mohit Agarwal
Is there any reason why cassandra doesn't do nodetool repair out of the box at some fixed intervals? On Tue, Aug 28, 2012 at 9:08 PM, Aaron Turner synfina...@gmail.com wrote: Funny you mention that... i just was hearing on #cassandra this morning that it repairs the replica set by default. I

Re: Automating nodetool repair

2012-08-28 Thread Edward Sargisson
Thanks a very nice approach. If every nodetool repair uses -pr does that satisfy the requirement to run a repair before GCGraceSeconds expires? In otherwords, will we get a correct result using -pr everywhere. Secondly, what's the need for sleep 120? Cheers, Edward On 12-08-28 07:03 AM,

Re: Automating nodetool repair

2012-08-28 Thread Aaron Turner
On Tue, Aug 28, 2012 at 1:42 PM, Edward Sargisson edward.sargis...@globalrelay.net wrote: Thanks a very nice approach. If every nodetool repair uses -pr does that satisfy the requirement to run a repair before GCGraceSeconds expires? In otherwords, will we get a correct result using -pr

Re: Automating nodetool repair

2012-08-28 Thread Omid Aladini
Secondly, what's the need for sleep 120? just give the cluster a chance to settle down between repairs... there's no real need for it, just is there because. Actually, repair could cause unreplicated data to be streamed and new sstables to be created. New sstables could cause pending

Automating nodetool repair

2012-08-27 Thread Edward Sargisson
Hi all, So nodetool repair has to be run regularly on all nodes. Does anybody have any interesting strategies or tools for doing this or is everybody just setting up cron to do it? For example, one could write some Puppet code to splay the cron times around so that only one should be running

Re: Automating nodetool repair

2012-08-27 Thread Aaron Turner
I use cron. On one box I just do: for n in node1 node2 node3 node4 ; do nodetool -h $n repair sleep 120 done A lot easier then managing a bunch of individual crontabs IMHO although I suppose I could of done it with puppet, but then you always have to keep an eye out that your repairs