Hi Jason - Did you run a major compaction after the repair completed? Do you have other reasons besides the number/size of sstables to believe all nodes don't have a copy of the current data at the end of the repair operation?
Thanks, Paul On Wed, Jun 8, 2016 at 8:12 AM, Jason Kania <jason.ka...@ymail.com> wrote: > Hi Romain, > > The problem is that there is no error to share. I am focusing on the > inconsistency that when I run nodetool repair, get no errors and yet the > content in the same directory on the different nodes is vastly different. > This lack of an error is nature of my question, not the nodetool compact > error. > > Thanks, > > Jason > > ------------------------------ > *From:* Romain Hardouin <romainh...@yahoo.fr> > *To:* "user@cassandra.apache.org" <user@cassandra.apache.org>; Jason > Kania <jason.ka...@ymail.com> > *Sent:* Wednesday, June 8, 2016 8:30 AM > *Subject:* Re: Nodetool repair inconsistencies > > Hi Jason, > > It's difficult for the community to help you if you don't share the error > ;-) > What the logs said when you ran a major compaction? (i.e. the first error > you encountered) > > Best, > > Romain > > Le Mercredi 8 juin 2016 3h34, Jason Kania <jason.ka...@ymail.com> a écrit > : > > > I am running a 3 node cluster of 3.0.6 instances and encountered an error > when running nodetool compact. I then ran nodetool repair. No errors were > returned. > > I then attempted to run nodetool compact again, but received the same > error so the repair made no correction and reported no errors. > > After that, I moved the problematic files out of the directory, restarted > cassandra and attempted the repair again. The repair again completed > without errors, however, no files were added to the directory that had > contained the corrupt files. So nodetool repair does not seem to be making > actual repairs. > > I started looking around and numerous directories have vastly different > amounts of content across the 3 nodes. There are 3 replicas so I would > expect to find similar amounts of content in the same data directory on the > different nodes. > > Is there any way to dig deeper into this? I don't want to be caught > because replication/repair is silently failing. I noticed that there is > always an "some repair failed" amongst the repair output but that is so > completely unhelpful and has always been present. > > Thanks, > > Jason > > > > >
