[
https://issues.apache.org/jira/browse/CASSANDRA-452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sammy Yu updated CASSANDRA-452:
-------------------------------
Description:
We noticed on one of our node the number of SStables is growing. The
compaction thread is alive and running. We can see that it is constantly
trying to compact the same set of sstables. However, it is failing because one
of the sstable is corrupt:
ERROR [ROW-READ-STAGE:475] 2009-09-21 00:29:17,068
DebuggableThreadPoolExecutor.java (line 125) Error in ThreadPoolExecutor
java.lang.RuntimeException: java.io.EOFException
at
org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:110)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:44)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.EOFException
at java.io.RandomAccessFile.readFully(RandomAccessFile.java:383)
at java.io.RandomAccessFile.readFully(RandomAccessFile.java:361)
at
org.apache.cassandra.utils.FBUtilities.readByteArray(FBUtilities.java:390)
at
org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:64)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:349)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:309)
at
org.apache.cassandra.db.filter.SSTableNamesIterator.<init>(SSTableNamesIterator.java:102)
at
org.apache.cassandra.db.filter.NamesQueryFilter.getSSTableColumnIterator(NamesQueryFilter.java:69)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1467)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1401)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1420)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1401)
at org.apache.cassandra.db.Table.getRow(Table.java:589)
at
org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:65)
at
org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:78)
... 4 more
was:
We noticed on one of our node the number of SStables is growing. It appears
that compaction thread is still alive. However, compaction is failing because
one of the sstable is corrupt:
ERROR [ROW-READ-STAGE:475] 2009-09-21 00:29:17,068
DebuggableThreadPoolExecutor.java (line 125) Error in ThreadPoolExecutor
java.lang.RuntimeException: java.io.EOFException
at
org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:110)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:44)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.EOFException
at java.io.RandomAccessFile.readFully(RandomAccessFile.java:383)
at java.io.RandomAccessFile.readFully(RandomAccessFile.java:361)
at
org.apache.cassandra.utils.FBUtilities.readByteArray(FBUtilities.java:390)
at
org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:64)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:349)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:309)
at
org.apache.cassandra.db.filter.SSTableNamesIterator.<init>(SSTableNamesIterator.java:102)
at
org.apache.cassandra.db.filter.NamesQueryFilter.getSSTableColumnIterator(NamesQueryFilter.java:69)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1467)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1401)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1420)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1401)
at org.apache.cassandra.db.Table.getRow(Table.java:589)
at
org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:65)
at
org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:78)
... 4 more
> Corrupt SSTable
> ---------------
>
> Key: CASSANDRA-452
> URL: https://issues.apache.org/jira/browse/CASSANDRA-452
> Project: Cassandra
> Issue Type: Bug
> Environment: Pre 0.4 based on r805615 on trunk w/ #370, #392, #394,
> #405, #406, #418
> Reporter: Sammy Yu
>
> We noticed on one of our node the number of SStables is growing. The
> compaction thread is alive and running. We can see that it is constantly
> trying to compact the same set of sstables. However, it is failing because
> one of the sstable is corrupt:
> ERROR [ROW-READ-STAGE:475] 2009-09-21 00:29:17,068
> DebuggableThreadPoolExecutor.java (line 125) Error in ThreadPoolExecutor
> java.lang.RuntimeException: java.io.EOFException
> at
> org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:110)
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:44)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.EOFException
> at java.io.RandomAccessFile.readFully(RandomAccessFile.java:383)
> at java.io.RandomAccessFile.readFully(RandomAccessFile.java:361)
> at
> org.apache.cassandra.utils.FBUtilities.readByteArray(FBUtilities.java:390)
> at
> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:64)
> at
> org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:349)
> at
> org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:309)
> at
> org.apache.cassandra.db.filter.SSTableNamesIterator.<init>(SSTableNamesIterator.java:102)
> at
> org.apache.cassandra.db.filter.NamesQueryFilter.getSSTableColumnIterator(NamesQueryFilter.java:69)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1467)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1401)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1420)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1401)
> at org.apache.cassandra.db.Table.getRow(Table.java:589)
> at
> org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:65)
> at
> org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:78)
> ... 4 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
