[ https://issues.apache.org/jira/browse/CASSANDRA-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12998901#comment-12998901 ]
Hudson commented on CASSANDRA-2206: ----------------------------------- Integrated in Cassandra-0.7 #316 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/316/]) improve detection and cleanup ofpartially-written sstables patch by jbellis; reviewed by gdusbabek for CASSANDRA-2206 > Startup fails due to cassandra trying to delete nonexisting file > ---------------------------------------------------------------- > > Key: CASSANDRA-2206 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2206 > Project: Cassandra > Issue Type: Bug > Components: Core > Affects Versions: 0.7.2 > Environment: linux > Reporter: Thibaut > Assignee: Jonathan Ellis > Fix For: 0.7.3 > > Attachments: 2206-v2.txt, 2206.txt > > Original Estimate: 1h > Remaining Estimate: 1h > > Hi, > On one of our nodes, startup fails due to cassandra trying to delete a > nonexistant "Data" file (see below). > Why that file is missing is another mistery... The log file entries don't > show any ERROR messages before cassandra restarted (for reasons I don't know) > and this error occured. > Directory listing: > total 109M > -rw-r--r-- 1 root root 51M 2011-02-21 05:25 table_task-f-1666-Data.db > -rw-r--r-- 1 root root 243K 2011-02-21 05:25 table_task-f-1666-Filter.db > -rw-r--r-- 1 root root 6.1M 2011-02-21 05:25 table_task-f-1666-Index.db > -rw-r--r-- 1 root root 4.2K 2011-02-21 05:25 table_task-f-1666-Statistics.db > -rw-r--r-- 1 root root 9.8M 2011-02-21 11:36 table_task-f-1703-Data.db > -rw-r--r-- 1 root root 57K 2011-02-21 11:36 table_task-f-1703-Filter.db > -rw-r--r-- 1 root root 1.3M 2011-02-21 11:36 table_task-f-1703-Index.db > -rw-r--r-- 1 root root 4.2K 2011-02-21 11:36 table_task-f-1703-Statistics.db > -rw-r--r-- 1 root root 292K 2011-02-21 11:42 table_task-f-1704-Data.db > -rw-r--r-- 1 root root 1.7K 2011-02-21 11:42 table_task-f-1704-Filter.db > -rw-r--r-- 1 root root 42K 2011-02-21 11:42 table_task-f-1704-Index.db > -rw-r--r-- 1 root root 4.2K 2011-02-21 11:42 table_task-f-1704-Statistics.db > -rw-r--r-- 1 root root 364K 2011-02-21 11:52 table_task-f-1705-Data.db > -rw-r--r-- 1 root root 2.0K 2011-02-21 11:52 table_task-f-1705-Filter.db > -rw-r--r-- 1 root root 50K 2011-02-21 11:52 table_task-f-1705-Index.db > -rw-r--r-- 1 root root 4.2K 2011-02-21 11:52 table_task-f-1705-Statistics.db > -rw-r--r-- 1 root root 535K 2011-02-21 12:10 table_task-f-1706-Data.db > -rw-r--r-- 1 root root 2.8K 2011-02-21 12:10 table_task-f-1706-Filter.db > -rw-r--r-- 1 root root 70K 2011-02-21 12:10 table_task-f-1706-Index.db > -rw-r--r-- 1 root root 4.2K 2011-02-21 12:10 table_task-f-1706-Statistics.db > -rw-r--r-- 1 root root 11M 2011-02-21 12:11 table_task-f-1707-Data.db > -rw-r--r-- 1 root root 18M 2011-02-21 09:47 table_task_meta-f-417-Data.db > -rw-r--r-- 1 root root 271K 2011-02-21 09:47 table_task_meta-f-417-Filter.db > -rw-r--r-- 1 root root 6.7M 2011-02-21 09:47 table_task_meta-f-417-Index.db > -rw-r--r-- 1 root root 4.2K 2011-02-21 09:47 > table_task_meta-f-417-Statistics.db > -rw-r--r-- 1 root root 1.2M 2011-02-21 10:47 table_task_meta-f-418-Data.db > -rw-r--r-- 1 root root 18K 2011-02-21 10:47 table_task_meta-f-418-Filter.db > -rw-r--r-- 1 root root 460K 2011-02-21 10:47 table_task_meta-f-418-Index.db > -rw-r--r-- 1 root root 4.2K 2011-02-21 10:47 > table_task_meta-f-418-Statistics.db > -rw-r--r-- 1 root root 791K 2011-02-21 11:47 table_task_meta-f-419-Data.db > -rw-r--r-- 1 root root 13K 2011-02-21 11:47 table_task_meta-f-419-Filter.db > -rw-r--r-- 1 root root 311K 2011-02-21 11:47 table_task_meta-f-419-Index.db > -rw-r--r-- 1 root root 4.2K 2011-02-21 11:47 > table_task_meta-f-419-Statistics.db > -rw-r--r-- 1 root root 57K 2011-02-21 12:11 table_task-tmp-f-1707-Filter.db > -rw-r--r-- 1 root root 1.4M 2011-02-21 12:11 table_task-tmp-f-1707-Index.db > -rw-r--r-- 1 root root 4.2K 2011-02-21 12:11 > table_task-tmp-f-1707-Statistics.db > Cassandra log: > /software/cassandra/bin/cassandra > rm: cannot remove `/software/cassandra/lib/jna.jar': No such file or directory > root@intr1n3:/cassandra/data/table_task# INFO 12:47:29,020 Logging > initialized > INFO 12:47:29,030 Heap size: 2614493184/2614493184 > INFO 12:47:29,031 JNA not found. Native methods will be disabled. > INFO 12:47:29,038 Loading settings from > file:/software/cassandra/conf/cassandra.yaml > INFO 12:47:29,320 DiskAccessMode is standard, indexAccessMode is mmap > INFO 12:47:29,332 Creating new commitlog segment > /hd1/cassandra_md5/commitlog/CommitLog-1298288849332.log > INFO 12:47:29,422 Opening /cassandra/data/system/Schema-f-244 > INFO 12:47:29,434 Opening /cassandra/data/system/Migrations-f-244 > INFO 12:47:29,437 Opening /cassandra/data/system/LocationInfo-f-137 > INFO 12:47:29,440 Opening /cassandra/data/system/HintsColumnFamily-f-352 > INFO 12:47:29,441 Opening /cassandra/data/system/HintsColumnFamily-f-353 > INFO 12:47:29,465 Loading schema version 54bc134e-2229-11e0-9159-fdf0b6b4b562 > WARN 12:47:29,623 Schema definitions were defined both locally and in > cassandra.yaml. Definitions in cassandra.yaml were ignored. > ERROR 12:47:29,638 Exception encountered during startup. > java.io.IOError: java.io.IOException: Failed to delete > /cassandra/data/table_task/table_task-tmp-f-1707-Data.db > at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145) > at > org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468) > at > org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153) > at > org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316) > at > org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79) > Caused by: java.io.IOException: Failed to delete > /cassandra/data/table_task/table_task-tmp-f-1707-Data.db > at > org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51) > at > org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41) > at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133) > ... 4 more > Exception encountered during startup. > java.io.IOError: java.io.IOException: Failed to delete > /cassandra/data/table_task/table_task-tmp-f-1707-Data.db > at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145) > at > org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468) > at > org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153) > at > org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316) > at > org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79) > Caused by: java.io.IOException: Failed to delete > /cassandra/data/table_task/table_task-tmp-f-1707-Data.db > at > org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51) > at > org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41) > at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133) > ... 4 more -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira