Try running nodetool scrub on the cf: its pretty good at detecting and fixing most corruption problems.
Dan -----Original Message----- From: Jonathan Colby [mailto:jonathan.co...@gmail.com] Sent: April-15-11 15:41 To: user@cassandra.apache.org Subject: recurring EOFException exception in 0.7.4 I've been struggling with these kinds of exceptions for some time now. I thought it might have been a one-time thing, so on the 2 nodes where I saw this problem I pulled in fresh data with a repair on an empty data directory. Unfortunately, this problem is now coming up on a new node that has, up until now, not had this problem. What could be causing this? Could it be related to encoding? Why are these rows not readable? This exception prevents cassandra from doing repairs, and even minor compactions. It also messes up memtable management (with a normal load of 25GB, disk goes to almost 100% full on a 500 GB hd). This is incredibly frustrating. This is the only pain-point I have had with cassandra so far. By the way, this node was never upgraded - it was 0.7.4 from the start, so that eliminates format compatibility problems. ERROR [CompactionExecutor:1] 2011-04-15 21:31:23,479 PrecompactedRow.java (line 82) Skipping row DecoratedKey(105452551814086725777389040553659117532, 4d657373616765456e726963686d656e743a313032343937) in /var/lib/cassandra/data/DFS/main-f-91-Data.db java.io.EOFException at java.io.RandomAccessFile.readFully(RandomAccessFile.java:383) at java.io.RandomAccessFile.readFully(RandomAccessFile.java:361) at org.apache.cassandra.io.util.BufferedRandomAccessFile.readBytes(BufferedRand omAccessFile.java:270) at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:315) at org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java :272) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:9 4) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:3 5) at org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumns(ColumnFami lySerializer.java:129) at org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithC olumns(SSTableIdentityIterator.java:176) at org.apache.cassandra.io.PrecompactedRow.<init>(PrecompactedRow.java:78) at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterato r.java:147) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.jav a:108) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.jav a:43) at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.jav a:73) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator .java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131 ) at org.apache.commons.collections.iterators.FilterIterator.setNextObject(Filter Iterator.java:183) at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterat or.java:94) at org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.jav a:449) at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:124) at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:94) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.ja va:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:9 08) at java.lang.Thread.run(Thread.java:662) No virus found in this incoming message. Checked by AVG - www.avg.com Version: 9.0.894 / Virus Database: 271.1.1/3574 - Release Date: 04/15/11 02:34:00
