I have also faced this issue. Rebooting the instance has been our fix so far. I am very interested if anyone else has a solution. I was unable to get a definitive answer from Datastax during the last Cassandra Summit.
From: Amit Singh F [mailto:amit.f.si...@ericsson.com] Sent: Thursday, June 30, 2016 7:02 AM To: user@cassandra.apache.org Subject: RE: C* files getting stuck Hi All, Please check fi anybody has faced below issue and if yes what best can be done to avoid this.? Thanks in advance. Regards Amit Singh From: Amit Singh F [mailto:amit.f.si...@ericsson.com] Sent: Wednesday, June 29, 2016 3:52 PM To: user@cassandra.apache.org<mailto:user@cassandra.apache.org> Subject: C* files getting stuck Hi All We are running Cassandra 2.0.14 and disk usage is very high. On investigating it further we found that there are around 4-5 files(~ 150 GB) in stuck mode. Command Fired : lsof /var/lib/cassandra | grep -i deleted Output : java 12158 cassandra 308r REG 8,16 34396638044 12727268 /var/lib/cassandra/data/mykeyspace/mycolumnfamily/mykeyspace-mycolumnfamily-jb-16481-Data.db (deleted) java 12158 cassandra 327r REG 8,16 101982374806 12715102 /var/lib/cassandra/data/mykeyspace/mycolumnfamily/mykeyspace-mycolumnfamily-jb-126861-Data.db (deleted) java 12158 cassandra 339r REG 8,16 12966304784 12714010 /var/lib/cassandra/data/mykeyspace/mycolumnfamily/mykeyspace-mycolumnfamily-jb-213548-Data.db (deleted) java 12158 cassandra 379r REG 8,16 15323318036 12714957 /var/lib/cassandra/data/mykeyspace/mycolumnfamily/mykeyspace-mycolumnfamily-jb-182936-Data.db (deleted) we are not able to see these files in any directory. This is somewhat similar to https://issues.apache.org/jira/browse/CASSANDRA-6275 which is fixed but still issue is there on higher version. Also in logs no error related to compaction is reported. so could any one of you please provide any suggestion how to counter this. Restarting Cassandra is one solution but this issue keeps on occurring so we cannot restart production machine is not recommended so frequently. Also we know that this version is not supported but there is high probability that it can occur in higher version too. Regards Amit Singh
