You should not run a truncate until the whole ring is reporting "Up/Normal." If there is a lot of flapping and it's a critical situation, disable hinted handoff as well (and you may want to move phi_convict_threshold up to 16 as well temporarily).
Stopping the compaction process temporarily on each node (it will restart as soon a memtable flushes again) has two positive effects for this case: - removes load and GC pressure - allows the truncate operation to complete fairly quick (as it is blocked by compaction) On Tue, May 26, 2015 at 8:33 AM, Ken Hancock <[email protected]> wrote: > Nate, how does this get around the issue? I'm guessing that just extends > the timeout, but if I had a server failure such that the server was down > for a couple hours, truncate would still have issues? > > > > On Sat, May 23, 2015 at 5:46 PM, Nate McCall <[email protected]> > wrote: > >> > >> > Truncate would have been the tool of choice, however my understanding >> is truncate fails unless all nodes are up and running which makes it a >> non-workable choice since we can't determine when failures will occur. >> > >> >> You can get around this via: >> - in cassandra.yaml, turning up "truncate_request_timeout_in_ms" to 10 >> minutes >> - stopping all compactions: nodetool stop >> [compaction|validation|cleanup|scrub|index_build] (that's 5 commands total) >> >> >> >> -- >> ----------------- >> Nate McCall >> Austin, TX >> @zznate >> >> Co-Founder & Sr. Technical Consultant >> Apache Cassandra Consulting >> http://www.thelastpickle.com >> > > > > > > -- ----------------- Nate McCall Austin, TX @zznate Co-Founder & Sr. Technical Consultant Apache Cassandra Consulting http://www.thelastpickle.com
