Re: nodetool repair failure

What is your GC_GRACE_SECONDS ? What kind repair option do you use for nodetool repair on a keyspace ? Did you start the repair on one node? did you use nodetool repair -pr ? or just "nodetool repair keyspace" ? How many nodetool repair processes do you use on the nodes? On Sun, Jul 30, 2017

Re: nodetool repair failure

On 2017-07-27 21:36 (-0700), Mitch Gitman wrote: > Now, the particular symptom to which that response refers is not what I was > seeing, but the response got me thinking that perhaps the failures I was > getting were on account of attempting to run "nodetool repair >

Re: nodetool repair failure

You need check the node that failed validation to find the relevant error. The IP should be in the logs of the node you started repair on. You shouldn't run multiple repairs on the same table from multiple nodes unless you really know what you're doing and not using vnodes. The failure you are

Re: nodetool repair failure

Michael, thanks for the input. I don't think I'm going to need to upgrade to 3.11 for the sake of getting nodetool repair working for me. Instead, I have another plausible explanation and solution for my particular situation. First, I should say that disk usage proved to be a red herring. There

Re: nodetool repair failure

On 07/27/2017 12:10 PM, Mitch Gitman wrote: > I'm using Apache Cassandra 3.10. > this is a dev cluster I'm talking about. > Further insights welcome... Upgrade and see if one of the many fixes for 3.11.0 helped? https://github.com/apache/cassandra/blob/cassandra-3.11.0/CHANGES.txt#L1-L129 If

Re: nodetool repair failure

I want to add an extra data point to this thread having encountered much the same problem. I'm using Apache Cassandra 3.10. I attempted to run an incremental repair that was optimized to take advantage of some downtime where the cluster is not fielding traffic and only repair each node's primary

RE: nodetool repair failure

: nodetool repair failure It did not help much. But other issue or error I saw when I repair the keyspace was it says "Sync failed between /xx.xx.xx.93 and /xx.xx.xx.94" this was run from .91 node. On Thu, Jun 29, 2017 at 4:44 PM, Akhil Mehra <akhilme...@gmail.com<mailto:akhi

Re: nodetool repair failure

It did not help much. But other issue or error I saw when I repair the keyspace was it says "Sync failed between /xx.xx.xx.93 and /xx.xx.xx.94" this was run from .91 node. On Thu, Jun 29, 2017 at 4:44 PM, Akhil Mehra wrote: > Run the following query and see if it gives

Re: nodetool repair failure

Run the following query and see if it gives you more information: select * from system_distributed.repair_history; Also is there any additional logging on the nodes where the error is coming from. Seems to be xx.xx.xx.94 for your last run. > On 30/06/2017, at 9:43 AM, Balaji Venkatesan

Re: nodetool repair failure

The verify and scrub went without any error on the keyspace. I ran it again with trace mode and still the same issue [2017-06-29 21:37:45,578] Parsing UPDATE system_distributed.parent_repair_history SET finished_at = toTimestamp(now()), successful_ranges = {'} WHERE

Re: nodetool repair failure

Balaji, Are you repairing a specific keyspace/table? if the failure is tied to a table, try 'verify' and 'scrub' options on .91...see if you get any errors. On Thursday, June 29, 2017, 12:12:14 PM PDT, Balaji Venkatesan wrote: Thanks. I tried with trace option

Re: nodetool repair failure

Thanks. I tried with trace option and there is not much info. Here are the few log lines just before it failed. [2017-06-29 19:01:54,969] /xx.xx.xx.93: Sending REPAIR_MESSAGE message to /xx.xx.xx.91 [2017-06-29 19:01:54,969] /xx.xx.xx.92: Appending to commitlog [2017-06-29 19:01:54,969]

Re: nodetool repair failure

nodetool repair has a trace option nodetool repair -tr yourkeyspacename see if that provides you with additional information. Regards, Akhil > On 28/06/2017, at 2:25 AM, Balaji Venkatesan > wrote: > > > We use Apache Cassandra 3.10-13 > > On Jun 26, 2017

Re: nodetool repair failure

We use Apache Cassandra 3.10-13 On Jun 26, 2017 8:41 PM, "Michael Shuler" wrote: What version of Cassandra? -- Michael On 06/26/2017 09:53 PM, Balaji Venkatesan wrote: > Hi All, > > When I run nodetool repair on a keyspace I constantly get "Some repair > failed"

Re: nodetool repair failure

What version of Cassandra? -- Michael On 06/26/2017 09:53 PM, Balaji Venkatesan wrote: > Hi All, > > When I run nodetool repair on a keyspace I constantly get "Some repair > failed" error, there are no sufficient info to debug more. Any help? > > Here is the stacktrace > >

nodetool repair failure

Hi All, When I run nodetool repair on a keyspace I constantly get "Some repair failed" error, there are no sufficient info to debug more. Any help? Here is the stacktrace == [2017-06-27 02:44:34,275] Some repair failed