[
https://issues.apache.org/jira/browse/CASSANDRA-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13004343#comment-13004343
]
Jonathan Ellis commented on CASSANDRA-2296:
-------------------------------------------
With debug logging turned on it looks like this:
{noformat}
[lots of rows around 119 bytes long]
DEBUG [CompactionExecutor:1] 2011-03-08 20:34:12,251 CompactionManager.java
(line 559) row 337a306f615f666c38756a is 119 bytes
DEBUG [CompactionExecutor:1] 2011-03-08 20:34:12,251 CompactionManager.java
(line 588) Index doublecheck: row 337a306f615f666c38756a is 119 bytes
DEBUG [CompactionExecutor:1] 2011-03-08 20:34:12,251 CompactionManager.java
(line 550) Reading row at 44385
DEBUG [CompactionExecutor:1] 2011-03-08 20:34:12,251 CompactionManager.java
(line 559) row 34306536785f666f666b65 is 0 bytes
DEBUG [CompactionExecutor:1] 2011-03-08 20:34:12,252 CompactionManager.java
(line 588) Index doublecheck: row 34306536785f666f666b65 is 0 bytes
WARN [CompactionExecutor:1] 2011-03-08 20:34:12,253 CompactionManager.java
(line 606) Non-fatal error reading row (stacktrace follows)
java.io.IOError: java.io.EOFException: bloom filter claims to be 734305 bytes,
longer than entire row size 0
at
org.apache.cassandra.io.sstable.SSTableIdentityIterator.<init>(SSTableIdentityIterator.java:125)
at
org.apache.cassandra.db.CompactionManager.doScrub(CompactionManager.java:597)
at
org.apache.cassandra.db.CompactionManager.access$600(CompactionManager.java:57)
at
org.apache.cassandra.db.CompactionManager$3.call(CompactionManager.java:196)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.io.EOFException: bloom filter claims to be 734305 bytes, longer
than entire row size 0
at
org.apache.cassandra.io.sstable.IndexHelper.defreezeBloomFilter(IndexHelper.java:113)
at
org.apache.cassandra.io.sstable.SSTableIdentityIterator.<init>(SSTableIdentityIterator.java:95)
... 8 more
WARN [CompactionExecutor:1] 2011-03-08 20:34:12,255 CompactionManager.java
(line 632) Row is unreadable; skipping to next
DEBUG [CompactionExecutor:1] 2011-03-08 20:34:12,255 CompactionManager.java
(line 550) Reading row at 44406
DEBUG [CompactionExecutor:1] 2011-03-08 20:34:12,255 CompactionManager.java
(line 559) row 34616465655f66707a6178 is 119 bytes
DEBUG [CompactionExecutor:1] 2011-03-08 20:34:12,255 CompactionManager.java
(line 588) Index doublecheck: row 34616465655f66707a6178 is 119 bytes
[lots more rows around 119 bytes]
{noformat}
In other words: there's an row that's empty except for the key, which is
causing the problem because we're not supposed to write rows like that. I
checked with a hex editor and that's what it looks like.
The good news is that scrub is correctly skipping it and recovering everything
else fine.
The bad news is we have (or possibly, had) a bug that was causing those empty
rows to be written.
> Scrub resulting in "bloom filter claims to be longer than entire row size"
> error
> --------------------------------------------------------------------------------
>
> Key: CASSANDRA-2296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2296
> Project: Cassandra
> Issue Type: Bug
> Reporter: Jason Harvey
> Attachments: sstable_part1.tar.bz2, sstable_part2.tar.bz2
>
>
> Doing a scrub on a node which I upgraded from 0.7.1 (was previously 0.6.8) to
> 0.7.3. Getting this error multiple times:
> {code}
> WARN [CompactionExecutor:1] 2011-03-08 18:33:52,513 CompactionManager.java
> (line 625) Row is unreadable; skipping to next
> WARN [CompactionExecutor:1] 2011-03-08 18:33:52,514 CompactionManager.java
> (line 599) Non-fatal error reading row (stacktrace follows)
> java.io.IOError: java.io.EOFException: bloom filter claims to be longer than
> entire row size
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.<init>(SSTableIdentityIterator.java:117)
> at
> org.apache.cassandra.db.CompactionManager.doScrub(CompactionManager.java:590)
> at
> org.apache.cassandra.db.CompactionManager.access$600(CompactionManager.java:56)
> at
> org.apache.cassandra.db.CompactionManager$3.call(CompactionManager.java:195)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.EOFException: bloom filter claims to be longer than entire
> row size
> at
> org.apache.cassandra.io.sstable.IndexHelper.defreezeBloomFilter(IndexHelper.java:113)
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.<init>(SSTableIdentityIterator.java:87)
> ... 8 more
> WARN [CompactionExecutor:1] 2011-03-08 18:33:52,515 CompactionManager.java
> (line 625) Row is unreadable; skipping to next
> INFO [CompactionExecutor:1] 2011-03-08 18:33:53,777 CompactionManager.java
> (line 637) Scrub of
> SSTableReader(path='/cassandra/data/reddit/Hide-f-671-Data.db') complete:
> 254709 rows in new sstable
> WARN [CompactionExecutor:1] 2011-03-08 18:33:53,777 CompactionManager.java
> (line 639) Unable to recover 1630 that were skipped. You can attempt manual
> recovery from the pre-scrub snapshot. You can also run nodetool repair to
> transfer the data from a healthy replica, if any
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
