[
https://issues.apache.org/jira/browse/CASSANDRA-10967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082542#comment-15082542
]
Terry Ma commented on CASSANDRA-10967:
--------------------------------------
nodetool status feeds
Datacenter: dc1
===============
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns (effective) Host ID
Rack
UN 172.31.20.223 665.85 GB 256 28.8%
e6c800d7-15ab-4b4b-8c0f-43a10df02fee rack4
UN 172.31.15.14 527.74 GB 256 28.8%
06a65c50-7a37-40df-9e05-7c84bc751e20 rack5
UN 172.31.8.188 436.05 GB 256 26.4%
b7dbfb00-6fe0-41b7-ae48-b38ba9fa95f9 rack1
UN 172.31.8.187 637.19 GB 256 28.5%
1666ea2e-fef1-42ef-9a66-de4d3c6e2704 rack2
UN 172.31.16.232 497.47 GB 256 29.2%
ad8ece0d-c2d9-4618-8c2f-e13e12ac110b rack6
UN 172.31.4.135 496.75 GB 256 28.7%
3c74e65f-93fd-4405-99b1-a5b9453d554c rack7
UJ 172.31.4.134 39.98 GB 256 ?
015f2a9a-a8d7-4c34-8936-184f35b56af6 rack8
UN 172.31.20.224 524.46 GB 256 29.6%
44743768-fc48-45ac-a0cd-6c9027e33afc rack3
nodetool netstats
Mode: JOINING
Bootstrap 366d1720-b376-11e5-9aac-bbecd825311e
/172.31.8.188
Receiving 50 files, 48498301504 bytes total. Already received 0 files,
0 bytes total
/172.31.20.223
Receiving 56 files, 80283672656 bytes total. Already received 0 files,
0 bytes total
/172.31.20.224
Receiving 49 files, 88426202890 bytes total. Already received 0 files,
0 bytes total
/172.31.16.232
Read Repair Statistics:
Attempted: 0
Mismatch (Blocking): 0
Mismatch (Background): 0
Pool Name Active Pending Completed
Large messages n/a 0 0
Small messages n/a 0 138949147
Gossip messages n/a 0 296937
> bootstrap fail with “Streaming error occurred”
> ----------------------------------------------
>
> Key: CASSANDRA-10967
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10967
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
> Environment: os: ubuntu 14.04.2 LTS (ec2)
> Reporter: Terry Ma
> Attachments: debug.log, system.log
>
>
> I installed cassandra 2.2.4 from datastax apt source
> I have a cassandra cluster in aws ec2. the cluster has 7 nodes now. and I
> want to add a new node to this cluster. but there is error occured with
> bootstrap. the bootstrap can not finish sucessfully.
> I got the following errors on the new node:
> {code}
> WARN [STREAM-IN-/172.31.4.135] 2016-01-01 15:23:44,228
> StreamSession.java:644 - [Stream #b1cc3600-b054-11e5-80c
> 4-21d1c7c11a01] Retrying for following error
> java.lang.ArrayIndexOutOfBoundsException: null
> ERROR [Thread-571] 2016-01-01 15:23:44,228 CassandraDaemon.java:185 -
> Exception in thread Thread[Thread-571,5,ma
> in]
> java.lang.RuntimeException: java.lang.InterruptedException
> at com.google.common.base.Throwables.propagate(Throwables.java:160)
> ~[guava-16.0.jar:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
> ~[apache-cassandra-2.2.4.jar:
> 2.2.4]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_66]
> Caused by: java.lang.InterruptedException: null
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer
> .java:1220) ~[na:1.8.0_66]
> at
> java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
> ~[na:1.8.0_66]
> at
> java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:350)
> ~[na:1.8.0_66]
> at
> org.apache.cassandra.streaming.compress.CompressedInputStream$Reader.runMayThrow(CompressedInputStrea
> m.java:176) ~[apache-cassandra-2.2.4.jar:2.2.4]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[apache-cassandra-2.2.4.jar:
> 2.2.4]
> {code}
> and I also got following errors:
> {code}
> WARN [STREAM-IN-/172.31.20.223] 2016-01-01 15:37:30,941
> StreamSession.java:644 - [Stream #b1cc3600-b054-11e5-80
> c4-21d1c7c11a01] Retrying for following error
> java.lang.RuntimeException: Last written key
> DecoratedKey(-8466153190082758358, 000000000000356a) >= current key
> DecoratedKey(-9223372036854775808, ) writing into
> /data_lvm/cassandra/data/feeds/inbox-eb873af0a19711e5ade0432b
> 31304f95/tmp-la-309-big-Data.db
> {code}
> I tried many things to do bootstrap, like cleanuped all nodes, scrubed all
> nodes, repaired all nodes. but It still fails with these errors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
