[
https://issues.apache.org/jira/browse/CASSANDRA-2283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13011360#comment-13011360
]
Jonathan Ellis commented on CASSANDRA-2283:
-------------------------------------------
IMO the right thing to do here is to deserialize enough of the data sent during
stream to (a) rewrite it to latest format and (b) write bloom filter and row
index -- currently this is done in a second pass post-stream.
> Streaming Old Format Data Fails in 0.7.3 after upgrade from 0.6.8
> -----------------------------------------------------------------
>
> Key: CASSANDRA-2283
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2283
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.3
> Environment: 0.7.3 upgraded from 0.6.8, Linux Java HotSpot(TM) 64-Bit
> Server VM (build 17.1-b03, mixed mode)
> Reporter: Erik Onnen
>
> After successfully upgrading a 0.6.8 ring to 0.7.3, we needed to bootstrap in
> a new node relatively quickly. When starting the new node with an assigned
> token in auto bootstrap mode, we see the following exceptions on the new node:
> INFO [main] 2011-03-07 10:37:32,671 StorageService.java (line 505) Joining:
> sleeping 30000 ms for pending range setup
> INFO [main] 2011-03-07 10:38:02,679 StorageService.java (line 505)
> Bootstrapping
> INFO [HintedHandoff:1] 2011-03-07 10:38:02,899 HintedHandOffManager.java
> (line 304) Started hinted handoff for endpoint /10.211.14.200
> INFO [HintedHandoff:1] 2011-03-07 10:38:02,900 HintedHandOffManager.java
> (line 360) Finished hinted handoff of 0 rows to endpoint /10.211.14.200
> INFO [CompactionExecutor:1] 2011-03-07 10:38:04,924 SSTableReader.java (line
> 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuff-f-1
> INFO [CompactionExecutor:1] 2011-03-07 10:38:05,390 SSTableReader.java (line
> 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuff-f-2
> INFO [CompactionExecutor:1] 2011-03-07 10:38:05,768 SSTableReader.java (line
> 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuffid-f-1
> INFO [CompactionExecutor:1] 2011-03-07 10:38:06,389 SSTableReader.java (line
> 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuffid-f-2
> INFO [CompactionExecutor:1] 2011-03-07 10:38:06,581 SSTableReader.java (line
> 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuffid-f-3
> ERROR [CompactionExecutor:1] 2011-03-07 10:38:07,056
> AbstractCassandraDaemon.java (line 114) Fatal exception in thread
> Thread[CompactionExecutor:1,1,main]
> java.io.EOFException
> at
> org.apache.cassandra.io.sstable.IndexHelper.skipIndex(IndexHelper.java:65)
> at
> org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:303)
> at
> org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:923)
> at
> org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:916)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> INFO [CompactionExecutor:1] 2011-03-07 10:38:08,480 SSTableReader.java (line
> 154) Opening /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuffid-f-5
> INFO [CompactionExecutor:1] 2011-03-07 10:38:08,582 SSTableReader.java (line
> 154) Opening
> /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuffid_reg_idx-f-1
> ERROR [CompactionExecutor:1] 2011-03-07 10:38:08,635
> AbstractCassandraDaemon.java (line 114) Fatal exception in thread
> Thread[CompactionExecutor:1,1,main]
> java.io.EOFException
> at
> org.apache.cassandra.io.sstable.IndexHelper.skipIndex(IndexHelper.java:65)
> at
> org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:303)
> at
> org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:923)
> at
> org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:916)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> ERROR [CompactionExecutor:1] 2011-03-07 10:38:08,666
> AbstractCassandraDaemon.java (line 114) Fatal exception in thread
> Thread[CompactionExecutor:1,1,main]
> java.io.EOFException
> at
> org.apache.cassandra.io.sstable.IndexHelper.skipIndex(IndexHelper.java:65)
> at
> org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:303)
> at
> org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:923)
> at
> org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:916)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> INFO [CompactionExecutor:1] 2011-03-07 10:38:08,855 SSTableReader.java (line
> 154) Opening
> /mnt/services/cassandra/var/data/0.7.3/data/Stuff/stuffid_reg_idx-f-4
> Two attempts to bootstrap in the new node both exhibited this behavior. On
> the node owning the tokens being migrated, stream activity is visible but
> doesn't update any progress I think due to the issues on the receiving host.
> Lastly, just case it's relevant, we had an EC2 node die underneath us during
> the upgrade so not all nodes were drained. This didn't affect the upgrade but
> I wanted to note it her to be thorough.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
