Re: Hundreds of sstables after every Repair

NTP output attached. Any other comments on the two queries ? Thanks Anuj Wadehra Sent from Yahoo Mail on Android From:Anuj Wadehra anujw_2...@yahoo.co.in Date:Tue, 9 Jun, 2015 at 10:59 pm Subject:Re: Hundreds of sstables after every Repair Yes. We use NTP. We also thought that drift is

Re: Hundreds of sstables after every Repair

*:Re: Hundreds of sstables after every Repair I think this came up recently in another thread. If you're getting large numbers of SSTables after repairs, that means that your nodes are diverging from the keys that they're supposed to be having. Likely you're dropping mutations. Do a nodetool

Re: Hundreds of sstables after every Repair

NTP output attached. Any other comments on the two queries ? Thanks Anuj Wadehra Sent from Yahoo Mail on Android From:Anuj Wadehra anujw_2...@yahoo.co.in Date:Tue, 9 Jun, 2015 at 10:59 pm Subject:Re: Hundreds of sstables after every Repair Yes. We use NTP. We also thought that drift is

Re: Hundreds of sstables after every Repair

NTP output attached. Any other comments on the two queries ? Thanks Anuj Wadehra Sent from Yahoo Mail on Android From:Anuj Wadehra anujw_2...@yahoo.co.in Date:Tue, 9 Jun, 2015 at 10:59 pm Subject:Re: Hundreds of sstables after every Repair Yes. We use NTP. We also thought that drift is

Re: Hundreds of sstables after every Repair

data? Thanks Anuj Wadehra Sent from Yahoo Mail on Android https://overview.mail.yahoo.com/mobile/?.src=Android -- *From*:Ken Hancock ken.hanc...@schange.com *Date*:Tue, 9 Jun, 2015 at 8:24 pm *Subject*:Re: Hundreds of sstables after every Repair I think

Re: Hundreds of sstables after every Repair

Any suggestions or comments on this one? Thanks Anuj Wadehra Sent from Yahoo Mail on Android From:Anuj Wadehra anujw_2...@yahoo.co.in Date:Sun, 7 Jun, 2015 at 1:54 am Subject:Hundreds of sstables after every Repair Hi, We are using 2.0.3 and vnodes. After every repair -pr operation  50+

Re: Hundreds of sstables after every Repair

I think this came up recently in another thread. If you're getting large numbers of SSTables after repairs, that means that your nodes are diverging from the keys that they're supposed to be having. Likely you're dropping mutations. Do a nodetool tpstats on each of your nodes and look at the

Re: Hundreds of sstables after every Repair

We were facing dropped mutations earlier and we increased flush writers. Now there are no dropped mutations in tpstats. To repair the damaged vnodes / inconsistent data we executed repair -pr on all nodes. Still, we see the same problem.  When we analyze repair logs we see 2 strange things:

Re: Hundreds of sstables after every Repair

Yes. We use NTP. We also thought that drift is creating problems. Our NTP Output is as under: [root@node1 ~]# ntpq -p remote   refid  st t when poll reach   delay   offset  jitter == +10.x.x.x