Just saw this weird assertion after upgrading one of my nodes from 0.8.6 to
1.0.2 (its been running fine for a few hours now):
INFO [FlushWriter:9] 2011-11-10 13:08:58,882 Memtable.java (line 237)
Writing Memtable-Data@1388955390(25676955/430716097 serialized/live bytes,
478913 ops)
ERROR [FlushWriter:9] 2011-11-10 13:08:59,513 AbstractCassandraDaemon.java
(line 133) Fatal exception in thread Thread[FlushWriter:9,5,main]
java.lang.AssertionError: CF size changed during serialization: was 4
initially but 3 written
at
org.apache.cassandra.db.ColumnFamilySerializer.serializeForSSTable(ColumnFam
ilySerializer.java:94)
at
org.apache.cassandra.db.ColumnFamilySerializer.serializeWithIndexes(ColumnFa
milySerializer.java:112)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:177)
at
org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:264)
at org.apache.cassandra.db.Memtable.access$400(Memtable.java:47)
at org.apache.cassandra.db.Memtable$4.runMayThrow(Memtable.java:289)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.ja
va:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:9
08)
at java.lang.Thread.run(Thread.java:662)
After taking a quick peek at the code, it looks like the numbers ("4
initially but 3 written") refer to the number of columns (in the memtable?).
Given the byte size of the memtable being flushed, there are *certainly*
more than 4 columns. Besides this error, there are no other unexpected or
unusual log entries and the node seems to be behaving normally.
Thoughts?
Dan Hendry
