[
https://issues.apache.org/jira/browse/CASSANDRA-8898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko resolved CASSANDRA-8898.
------------------------------------------
Resolution: Cannot Reproduce
We've had many changes to that code in the last few releases, and it's likely
have been resolved since then. If you can reproduce in latest 2.2.x or 3.0.x,
please let us know, and reopen the JIRA.
> sstableloader utility should allow loading of data from mounted filesystem
> --------------------------------------------------------------------------
>
> Key: CASSANDRA-8898
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8898
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
> Environment: 2.0.12
> Reporter: Kenneth Failbus
>
> When trying to load data from a mounted filesystem onto a new cluster,
> following exceptions is observed intermittently, and at some point the
> sstableloader process gets hung without completing the loading process.
> Please note that in my case the scenario was loading the existing sstables
> from an existing cluster to a brand new cluster.
> Finally, it was found that there were some hard assumptions been made by
> sstableloader utility w.r.t response from the filesystem, which was not
> working with mounted filesystem.
> The work-around was to copy each existing nodes sstable data files locally
> and then point sstableloader to that local filesystem to then load data onto
> new cluster.
> In case of restoring during disaster recovery from backups the data using
> sstableloader, this copying to local filesystem of data files and then
> loading would take a long time.
> It would be a good enhancement of the sstableloader utility to enable use of
> mounted filesystem as copying data locally and then loading is time consuming.
> Below is the exception seen during the use of mounted filesystem.
> {code}
> java.lang.AssertionError: Reference counter -1 for
> /opt/tmp/casapp-c1-c00053-g.ch.tvx.comcast.com/MinDataService/System/MinDataService-System-jb-5449-Data.db
>
> at
> org.apache.cassandra.io.sstable.SSTableReader.releaseReference(SSTableReader.java:1146)
>
> at
> org.apache.cassandra.streaming.StreamTransferTask.complete(StreamTransferTask.java:74)
>
> at
> org.apache.cassandra.streaming.StreamSession.received(StreamSession.java:542)
> at
> org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:424)
>
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:245)
>
> at java.lang.Thread.run(Thread.java:744)
> WARN 21:07:16,853 [Stream #3e5a5ba0-bdef-11e4-a975-5777dbff0945] Stream failed
> at
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:59)
>
> at
> org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:1406)
>
> at
> org.apache.cassandra.streaming.compress.CompressedStreamWriter.write(CompressedStreamWriter.java:55)
>
> at
> org.apache.cassandra.streaming.messages.OutgoingFileMessage$1.serialize(OutgoingFileMessage.java:59)
>
> at
> org.apache.cassandra.streaming.messages.OutgoingFileMessage$1.serialize(OutgoingFileMessage.java:42)
>
> at
> org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:45)
>
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:339)
>
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:311)
>
> at java.lang.Thread.run(Thread.java:744)
> Caused by: java.io.FileNotFoundException:
> /opt/tmp/casapp-c1-c00055-g.ch.tvx.comcast.com/MinDataService/System/MinDataService-System-jb-5997-Data.db
> (No such file or directory)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:241)
> at
> org.apache.cassandra.io.util.RandomAccessReader.<init>(RandomAccessReader.java:58)
>
> at
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.<init>(CompressedRandomAccessReader.java:76)
>
> at
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:55)
>
> ... 8 more
> Exception in thread "STREAM-OUT-/96.115.88.196"
> java.lang.NullPointerException
> at
> org.apache.cassandra.streaming.ConnectionHandler$MessageHandler.signalCloseDone(ConnectionHandler.java:205)
>
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:331)
>
> at java.lang.Thread.run(Thread.java:744)
> ERROR 20:49:35,646 [Stream #d9fce650-bdf3-11e4-b6c0-252cb9b3e9f3] Streaming
> error occurred
> java.lang.AssertionError: Reference counter -3 for
> /opt/tmp/casapp-c1-c00055-g.ch.tvx.comcast.com/MinDataService/System/MinDataService-System-jb-4897-Data.db
>
> at
> org.apache.cassandra.io.sstable.SSTableReader.releaseReference(SSTableReader.java:1146)
>
> at
> org.apache.cassandra.streaming.StreamTransferTask.complete(StreamTransferTask.java:74)
>
> at
> org.apache.cassandra.streaming.StreamSession.received(StreamSession.java:542)
> at
> org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:424)
>
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:245)
>
> at java.lang.Thread.run(Thread.java:744)
> Exception in thread "STREAM-IN-/96.115.88.196" java.lang.NullPointerException
> at
> org.apache.cassandra.streaming.ConnectionHandler$MessageHandler.signalCloseDone(ConnectionHandler.java:205)
>
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:260)
>
> at java.lang.Thread.run(Thread.java:744)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
