There have also been a few sstable ref counting bugs that would over report load in nodetool ring/status due to overlapping normal and incremental repairs (which you should probably avoid doing anyway)
-- Jeff Jirsa > On Apr 19, 2018, at 9:27 AM, Rahul Singh <rahul.xavier.si...@gmail.com> wrote: > > I’ve seen something similar in 2.1. Our issue was related to file permissions > being flipped due to an automation and C* stopped seeing Sstables so it > started making new data — via read repair or repair processes. > > In your case if nodetool is reporting data that means that it’s growing due > to data growth. What does your cfstats / tablestats day? Are you monitoring > your key tables data via cfstats metrics like SpaceUsedLive or > SpaceUsedTotal. What is your snapshottjng / backup process doing? > > -- > Rahul Singh > rahul.si...@anant.us > > Anant Corporation > >> On Apr 19, 2018, 7:01 AM -0500, horschi <hors...@gmail.com>, wrote: >> Did you check the number of files in your data folder before & after the >> restart? >> >> I have seen cases where cassandra would keep creating sstables, which >> disappeared on restart. >> >> regards, >> Christian >> >> >>> On Thu, Apr 19, 2018 at 12:18 PM, Fernando Neves <fernando1ne...@gmail.com> >>> wrote: >>> I am facing one issue with our Cassandra cluster. >>> >>> Details: Cassandra 3.0.14, 12 nodes, 7.4TB(JBOD) disk size in each node, >>> ~3.5TB used physical data in each node, ~42TB whole cluster and default >>> compaction setup. This size maintain the same because after the retention >>> period some tables are dropped. >>> >>> Issue: Nodetool status is not showing the correct used size in the output. >>> It keeps increasing the used size without limit until automatically node >>> shutdown or until our sequential scheduled restart(workaround 3 times >>> week). After the restart, nodetool shows the correct used space but for few >>> days. >>> Did anybody have similar problem? Is it a bug? >>> >>> Stackoverflow: >>> https://stackoverflow.com/questions/49668692/cassandra-nodetool-status-is-not-showing-correct-used-space >>> >>
