[
https://issues.apache.org/jira/browse/CASSANDRA-6646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935373#comment-13935373
]
sankalp kohli edited comment on CASSANDRA-6646 at 3/14/14 6:17 PM:
-------------------------------------------------------------------
Here is the stack trace. It is CorruptSSTableException which is a problem.
ERROR [ReadStage:2257] xxxxx CassandraDaemon.java (line 191) Exception in
thread Thread[ReadStage:2257,5,main]
org.apache.cassandra.io.sstable.CorruptSSTableException:
org.apache.cassandra.io.compress.CorruptBlockException:
(/xx/data/Cassandra/xxxxx/xxxxx/xxxxxxx-166-Data.db): corruption detected,
chunk at 148883342 of length 23271.
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBuffer(CompressedRandomAccessReader.java:89)
at
org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:312)
at
org.apache.cassandra.io.util.PoolingSegmentedFile.getSegment(PoolingSegmentedFile.java:42)
at
org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:1048)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.<init>(SimpleSliceReader.java:63)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:68)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SSTableSliceIterator.java:44)
at
org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:104)
at
org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:68)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:272)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:65)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1391)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1207)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1123)
at org.apache.cassandra.db.Table.getRow(Table.java:347)
at
org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:70)
at
org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:44)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.cassandra.io.compress.CorruptBlockException:
(xxxx-Data.db): corruption detected, chunk at 148883342 of length 23271.
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.decompressChunk(CompressedRandomAccessReader.java:120)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBuffer(CompressedRandomAccessReader.java:85)
... 20 more
was (Author: kohlisankalp):
Here is the stack trace
ERROR [ReadStage:2257] xxxxx CassandraDaemon.java (line 191) Exception in
thread Thread[ReadStage:2257,5,main]
org.apache.cassandra.io.sstable.CorruptSSTableException:
org.apache.cassandra.io.compress.CorruptBlockException:
(/xx/data/Cassandra/xxxxx/xxxxx/xxxxxxx-166-Data.db): corruption detected,
chunk at 148883342 of length 23271.
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBuffer(CompressedRandomAccessReader.java:89)
at
org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:312)
at
org.apache.cassandra.io.util.PoolingSegmentedFile.getSegment(PoolingSegmentedFile.java:42)
at
org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:1048)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.<init>(SimpleSliceReader.java:63)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:68)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SSTableSliceIterator.java:44)
at
org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:104)
at
org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:68)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:272)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:65)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1391)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1207)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1123)
at org.apache.cassandra.db.Table.getRow(Table.java:347)
at
org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:70)
at
org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:44)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.cassandra.io.compress.CorruptBlockException:
(xxxx-Data.db): corruption detected, chunk at 148883342 of length 23271.
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.decompressChunk(CompressedRandomAccessReader.java:120)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBuffer(CompressedRandomAccessReader.java:85)
... 20 more
> Disk Failure Policy ignores CorruptBlockException
> --------------------------------------------------
>
> Key: CASSANDRA-6646
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6646
> Project: Cassandra
> Issue Type: Bug
> Reporter: sankalp kohli
> Assignee: Marcus Eriksson
> Priority: Minor
> Fix For: 2.0.7, 2.1 beta2
>
> Attachments:
> 0001-make-CorruptSSTableException-an-FSError-to-be-able-t.patch
>
>
> If Cassandra is using compression and has a bad drive or stable, it will
> throw an CorruptBlockException.
> Disk Failure Policy only works if it is an FSError and does not work for
> IOExceptions like this.
> We need to better handle such exceptions as it causes nodes to not respond to
> the co-ordinator causing the client to timeout.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
