Re: how to monitor nodetool cleanup?

2013-05-09 Thread aaron morton
nodetool setcompactionthroughput controls the speed of compaction, and cleanup 
runs in the compaction manager. 

Cheers

-
Aaron Morton
Freelance Cassandra Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 8/05/2013, at 8:59 AM, Michael Morris michael.m.mor...@gmail.com wrote:

 Not sure about making things go faster, but you should be able to monitor it 
 with nodetool compactionstats.
 
 Thanks,
 
 Mike
 
 
 On Tue, May 7, 2013 at 12:43 PM, Brian Tarbox tar...@cabotresearch.com 
 wrote:
 I'm recovering from a significant failure and so am doing lots of nodetool 
 move, removetoken, repair and cleanup.
 
 For most of these I can do nodetool netstats to monitor progress but it 
 doesn't show anything for cleanup...how can I monitor the progress of 
 cleanup?  On a related note: I'm able to stop all client access to the 
 cluster until things are happy again...is there anything I can do to make 
 move/repair/cleanup go faster?
 
 FWIW my problems came from trying to move nodes between EC2 availability 
 zones...which led to
 1) killing a node and recreating it in another availability zone
 2) new node had different local ip address so cluster thought old node was 
 just down and we had a new node...
 
 I did the removetoken on the dead node and gave the new node oldToken-1...but 
 things still got weird and I ended up spending a couple of days cleaning up 
 (which seems odd for only about 300 gig total data).
 
 Anyway, any suggestions for monitoring / speeding up cleanup would be 
 appreciated.
 
 Brian Tarbox
 
 



how to monitor nodetool cleanup?

2013-05-07 Thread Brian Tarbox
I'm recovering from a significant failure and so am doing lots of nodetool
move, removetoken, repair and cleanup.

For most of these I can do nodetool netstats to monitor progress but it
doesn't show anything for cleanup...how can I monitor the progress of
cleanup?  On a related note: I'm able to stop all client access to the
cluster until things are happy again...is there anything I can do to make
move/repair/cleanup go faster?

FWIW my problems came from trying to move nodes between EC2 availability
zones...which led to
1) killing a node and recreating it in another availability zone
2) new node had different local ip address so cluster thought old node was
just down and we had a new node...

I did the removetoken on the dead node and gave the new node
oldToken-1...but things still got weird and I ended up spending a couple of
days cleaning up (which seems odd for only about 300 gig total data).

Anyway, any suggestions for monitoring / speeding up cleanup would be
appreciated.

Brian Tarbox


Re: how to monitor nodetool cleanup?

2013-05-07 Thread Michael Morris
Not sure about making things go faster, but you should be able to monitor
it with nodetool compactionstats.

Thanks,

Mike


On Tue, May 7, 2013 at 12:43 PM, Brian Tarbox tar...@cabotresearch.comwrote:

 I'm recovering from a significant failure and so am doing lots of nodetool
 move, removetoken, repair and cleanup.

 For most of these I can do nodetool netstats to monitor progress but it
 doesn't show anything for cleanup...how can I monitor the progress of
 cleanup?  On a related note: I'm able to stop all client access to the
 cluster until things are happy again...is there anything I can do to make
 move/repair/cleanup go faster?

 FWIW my problems came from trying to move nodes between EC2 availability
 zones...which led to
 1) killing a node and recreating it in another availability zone
 2) new node had different local ip address so cluster thought old node was
 just down and we had a new node...

 I did the removetoken on the dead node and gave the new node
 oldToken-1...but things still got weird and I ended up spending a couple of
 days cleaning up (which seems odd for only about 300 gig total data).

 Anyway, any suggestions for monitoring / speeding up cleanup would be
 appreciated.

 Brian Tarbox