Thanks Jeff, I have a C* cluster spread across multiple datacenter. reason for cleanup : I added multiple nodes to cluster and need to run cleanup on old nodes so that the redundant data is cleaned-up.
On Thu, May 11, 2017 at 11:08 AM, Jeff Jirsa <jji...@apache.org> wrote: > > > On 2017-05-10 22:44 (-0700), Jai Bheemsen Rao Dhanwada < > jaibheem...@gmail.com> wrote: > > Hello, > > > > I am running into an issue where *nodetool cleanup *fails to cleanup > data. > > We are running 2.1.16 version of Cassandra. > > > > > > [user@host ~]$ nodetool cleanup > > Aborted cleaning up atleast one column family in keyspace user, check > > server logs for more information. > > Aborted cleaning up atleast one column family in keyspace org, check > server > > logs for more information. > > error: nodetool failed, check server logs > > -- StackTrace -- > > java.lang.RuntimeException: nodetool failed, check server logs > > at > > org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:294) > > at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:206) > > > > *Logs:* > > > > INFO [RMI TCP Connection(17)-x.x.x.x] 2017-05-05 04:04:07,987 > > CompactionManager.java:415 - Cleanup cannot run before a node has joined > > the ring > > INFO [RMI TCP Connection(17)-x.x.x.x] 2017-05-05 04:04:08,010 > > CompactionManager.java:415 - Cleanup cannot run before a node has joined > > the ring > > > > All the nodes in the cluster are up and running. We tried doing a rolling > > restart of all nodes and no luck. > > > > After looking at the Cassandra JIRA : > > https://issues.apache.org/jira/browse/CASSANDRA-10991 looks like the > issue > > is fixed with 2.2.6 and 3.0 version. > > While we have plans to upgrade to the latest versions(which might take > > longer time), does any know if there is any work around to mitigate the > > issue? > > > > Are you running multiple datacenters, and you just removed a specific > datacenter from a keyspace (and that's why you want to run cleanup)? If > that's the case, I fear the fix for 10991 isn't really going to fix it in > the way you hope (we may need a follow-up jira). What you'll almost > certainly need to do is remove the data on disk manually, which is quite > unfortunate as it'll require you to stop+delete-data-for-that-keyspace+start > each node in the datacenter for which you removed replication. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org > For additional commands, e-mail: user-h...@cassandra.apache.org > >
