The commitlog growing is often a symptom of a problem. If the memtable flush or post flush fails in anyway, the commitlogs will not be recycled/deleted and will continue to pool up.
Might want to go back in logs earlier to make sure theres nothing like the postmemtable flusher getting a permission error (some tooling creates commitlogs so if run by wrong user can create this prooblem), or a memtable flush error. You can also check tpstats to see if tasks are queued up in postmemtable flusher and jstack to see where the active ones are stuck if they are. Chris > On Jan 30, 2018, at 4:20 AM, Amit Singh <dev...@nic.in> wrote: > > Hi, > > When you actually say nodetool flush, data from memTable goes to disk based > structure as SStables and side by side , commit logs segments for that > particular data get written off and its continuous process . May be in your > case , you can decrease the value of below uncommented property in > Cassandra.yaml > > commitlog_total_space_in_mb > > Also this is what is it used for > > # Total space to use for commit logs on disk. > # > # If space gets above this value, Cassandra will flush every dirty CF > # in the oldest segment and remove it. So a small total commitlog space > # will tend to cause more flush activity on less-active columnfamilies. > # > # The default value is the smaller of 8192, and 1/4 of the total space > # of the commitlog volume. > > > From: Mokkapati, Bhargav (Nokia - IN/Chennai) > [mailto:bhargav.mokkap...@nokia.com] > Sent: Tuesday, January 30, 2018 4:00 PM > To: user@cassandra.apache.org > Subject: Commitlogs are filling the Full Disk space and nodes are down > > Hi Team, > > My Cassandra version : Apache Cassandra 3.0.13 > > Cassandra nodes are down due to Commitlogs are getting filled up until full > disk size. > > <image001.jpg> > > With “Nodetool flush” I didn’t see any commitlogs deleted. > > Can anyone tell me how to flush the commitlogs without losing data. > > Thanks, > Bhargav M
