[
https://issues.apache.org/jira/browse/CASSANDRA-11583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15243634#comment-15243634
]
Jens Rantil edited comment on CASSANDRA-11583 at 4/15/16 9:26 PM:
------------------------------------------------------------------
Hm, rereading
https://github.com/apache/cassandra/blob/cassandra-2.1.13/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java#L208,
it looks like the assert is only made when the streaming is done, right? Can I
be comfortable that `sstableloader` finished succesfully if it doesn't print
any error before the exception (and the assert fails in "onSuccess" method)?
was (Author: ztyx):
Hm, rereading
https://github.com/apache/cassandra/blob/cassandra-2.1.13/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java#L208,
it looks like the assert is only made when the streaming is done, right? Can I
be comfortable that `sstableloader` finished succesfully if it doesn't print
any error before the exception?
> Exception when streaming sstables using `sstableloader`
> -------------------------------------------------------
>
> Key: CASSANDRA-11583
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11583
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: $ uname -a
> Linux bigdb-100 3.2.0-99-virtual #139-Ubuntu SMP Mon Feb 1 23:52:21 UTC 2016
> x86_64 x86_64 x86_64 GNU/Linux
> I am using Datastax Enterprise 4.7.8-1 which is based on 2.1.13.
> Reporter: Jens Rantil
>
> This bug came out of CASSANDRA-11562.
> I have have a keyspace snapshotted from a 2.1.11 (DSE 4.7.5-1) node. When I'm
> running the `sstableloader` I get the following output/exception:
> {noformat}
> # sstableloader --nodes X.X.X.20 --username YYY --password ZZZ --ignore XXX
> /var/lib/cassandra/data/XXX/ZZZ-f7ebdf0daa3a3062828fddebc109a3b2
> Established connection to initial hosts
> Opening sstables and calculating sections to stream
> Streaming relevant part of
> /var/lib/cassandra/data/XXX/ZZZ-f7ebdf0daa3a3062828fddebc109a3b2/XXX-ZZZ-ka-6463-Data.db
>
> /var/lib/cassandra/data/XXX/ZZZ-f7ebdf0daa3a3062828fddebc109a3b2/tink-ZZZ-ka-6464-Data.db
> to [/X.X.X.33, /X.X.X.113, /X.X.X.32, /X.X.X.20, /X.X.X.122, /X.X.X.176,
> /X.X.X.143, /X.X.X.172, /X.X.X.50, /X.X.X.51, /X.X.X.52, /X.X.X.71,
> /X.X.X.53, /X.X.X.54, /X.X.X.47, /X.X.X.31, /X.X.X.8]
> progress: [/X.X.X.113]0:0/2 0 % [/X.X.X.143]0:0/2 0 % [/X.X.X.172]0:0/2 0
> % [/X.X.X.20]0:0/2 0 % [/X.X.X.71]0:0/2 0 % [/X.X.X.122]0:0/2 0 %
> [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 0 % [/X.X.X.143]0:0/2 0 %
> [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:0/2 0 %
> [/X.X.X.122]0:0/2 0 % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 0 %
> [/X.X.X.143]0:0/2 1 % [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 %
> [/X.X.X.71]0:0/2 0 % [/X.X.X.122]0:0/2 0 % [/X.X.X.47]0:0/2 progress:
> [/X.X.X.113]0:0/2 0 % [/X.X.X.143]0:1/2 1 % [/X.X.X.172]0:0/2 0 %
> [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:0/2 0 % [/X.X.X.122]0:0/2 0 %
> [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 0 % [/X.X.X.143]0:1/2 1 %
> [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:1/2 1 %
> [/X.X.X.122]0:0/2 0 % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 0 %
> [/X.X.X.143]0:1/2 1 % [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 %
> [/X.X.X.71]0:1/2 1 % [/X.X.X.122]0:1/2 1 % [/X.X.X.47]0:0/2 progress:
> [/X.X.X.113]0:0/2 0 % [/X.X.X.143]0:1/2 1 % [/X.X.X.172]0:0/2 0 %
> [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:1/2 1 % [/X.X.X.122]0:1/2 1 %
> [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 7 % [/X.X.X.143]0:1/2 1 %
> [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:1/2 1 %
> [/X.X.X.122]0:1/2 1 % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 7 %
> [/X.X.X.143]0:1/2 6 % [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 %
> [/X.X.X.71]0:1/2 1 % [/X.X.X.122]0:1/2 1 % [/X.X.X.47]0:0/2 progress:
> [/X.X.X.113]0:0/2 12 % [/X.X.X.143]0:1/2 6 % [/X.X.X.172]0:0/2 0 %
> [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:1/2 1 % [/X.X.X.122]0:1/2 1 %
> [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 12 % [/X.X.X.143]0:1/2 11 %
> [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:1/2 1 %
> [/X.X.X.122]0:1/2 1 % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 19 %
> [/X.X.X.143]0:1/2 11 % [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 %
> [/X.X.X.71]0:1/2 1 % [/X.X.X.122]0:1/2 1 % [/X.X.X.47]0:0/2 progress:
> [/X.X.X.113]0:0/2 19 % [/X.X.X.143]0:1/2 15 % [/X.X.X.172]0:0/2 0 %
> [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:1/2 1 % [/X.X.X.122]0:1/2 1 %
> [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 26 % [/X.X.X.143]0:1/2 15 %
> [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:1/2 1 %
> [/X.X.X.122]0:1/2 1 % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 26 %
> [/X.X.X.143]0:1/2 20 % [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 1 %
> [/X.X.X.71]0:1/2 1 % [/X.X.X.122]0:1/2 1 % [/X.X.X.47]0:0/2 progress:
> [/X.X.X.113]0:0/2 26 % [/X.X.X.143]0:1/2 21 % [/X.X.X.172]0:0/2 0 %
> [/X.X.X.20]0:1/2 1 % [/X.X.X.71]0:1/2 1 % [/X.X.X.122]0:1/2 1 %
> [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 26 % [/X.X.X.143]0:1/2 21 %
> [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 3 % [/X.X.X.71]0:1/2 1 %
> [/X.X.X.122]0:1/2 1 % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 42 %
> [/X.X.X.143]0:1/2 27 % [/X.X.X.172]0:0/2 0 % [/X.X.X.20]0:1/2 3 %
> [/X.X.X.71]0:1/2 6 % [/X.X.X.122]0:1/2 1 % [/X.X.X.47]0:0/2
> [...]
> progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 78
> % [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:1/2 97 %
> [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100%
> [/X.X.X.172]0:0/2 78 % [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100%
> [/X.X.X.122]0:1/2 97 % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:2/2 100%
> [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 86 % [/X.X.X.20]0:2/2 100%
> [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:0/2 progress:
> [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 86 %
> [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100%
> [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100%
> [/X.X.X.172]0:0/2 86 % [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100%
> [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:2/2 100%
> [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 86 % [/X.X.X.20]0:2/2 100%
> [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:0/2 progress:
> [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 86 %
> [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100%
> [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100%
> [/X.X.X.172]0:0/2 95 % [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100%
> [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:2/2 100%
> [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 95 % [/X.X.X.20]0:2/2 100%
> [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:1/2 progress:
> [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 95 %
> [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100%
> [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100%
> [/X.X.X.172]0:0/2 95 % [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100%
> [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100%
> [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:1/2 98 % [/X.X.X.20]0:2/2 100%
> [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress:
> [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:1/2 99 %
> [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100%
> [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100%
> [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100%
> [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100%
> [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100%
> [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress:
> [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:2/2 100%
> [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100%
> [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100%
> [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100%
> [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100%
> [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100%
> [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress:
> [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:2/2 100%
> [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100%
> [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100%
> [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100%
> [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100%
> [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100%
> [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress:
> [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:2/2 100%
> [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100%
> [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100%
> [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100%
> [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 100% [/X.X.X.176]0:2/2 100%
> [/X.X.X.8]0:2/2 100% total: 100% 0 MB/s(avg: 23 MB/s)
> Summary statistics:
> Connections per host: : 1
> Total files transferred: : 14
> Total bytes transferred: : 142070241
> Total duration (ms): : 5823
> Average transfer rate (MB/s): : 23
> Peak transfer rate (MB/s): : 33
> ERROR 09:07:12 [Stream #6e345de0-02e9-11e6-b944-1fee10beb91b] Streaming error
> occurred
> java.lang.AssertionError: null
> at
> org.apache.cassandra.io.sstable.SSTableLoader.releaseReferences(SSTableLoader.java:208)
> ~[cassandra-all-2.1.13.1218.jar:2.1.13.1218]
> at
> org.apache.cassandra.io.sstable.SSTableLoader.onSuccess(SSTableLoader.java:193)
> ~[cassandra-all-2.1.13.1218.jar:2.1.13.1218]
> at
> org.apache.cassandra.io.sstable.SSTableLoader.onSuccess(SSTableLoader.java:48)
> ~[cassandra-all-2.1.13.1218.jar:2.1.13.1218]
> at com.google.common.util.concurrent.Futures$4.run(Futures.java:1181)
> ~[guava-16.0.1.jar:na]
> at
> com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
> ~[guava-16.0.1.jar:na]
> at
> com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
> ~[guava-16.0.1.jar:na]
> at
> com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
> ~[guava-16.0.1.jar:na]
> at
> com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:185)
> ~[guava-16.0.1.jar:na]
> at
> org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:213)
> ~[cassandra-all-2.1.13.1218.jar:2.1.13.1218]
> at
> org.apache.cassandra.streaming.StreamResultFuture.handleSessionComplete(StreamResultFuture.java:184)
> ~[cassandra-all-2.1.13.1218.jar:2.1.13.1218]
> at
> org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:415)
> ~[cassandra-all-2.1.13.1218.jar:2.1.13.1218]
> at
> org.apache.cassandra.streaming.StreamSession.complete(StreamSession.java:607)
> ~[cassandra-all-2.1.13.1218.jar:2.1.13.1218]
> at
> org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:471)
> ~[cassandra-all-2.1.13.1218.jar:2.1.13.1218]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:263)
> ~[cassandra-all-2.1.13.1218.jar:2.1.13.1218]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]
> {noformat}
> Question: Is it an issue that I dumped the sstables using a _slightly_ older
> Cassandra node (2.1.11 instead of 2.1.13)? Is streaming from 2.1.13 to 2.1.11
> incompatible?
> Let me know if you need anything else from me. Also, know of any possible for
> workaround?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
