[
https://issues.apache.org/jira/browse/CASSANDRA-10291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14740585#comment-14740585
]
Ara Sadoyan commented on CASSANDRA-10291:
-----------------------------------------
ok I have cleared all logs and did bootstrap of Cassandra node. Attached full
system.log during last bootstrap.
To have full picture I have also attached nodetool netstats and one hanged
stream's stack trace.
Stack trace is for only one stream, but all other hanged ones are almost the
same.
There were no traffic and working streams when I was collecting date.
> Bootstrap hangs on adding new node
> -----------------------------------
>
> Key: CASSANDRA-10291
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10291
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Debian 7 64 bit
> HotSpot JDK 1.7.0_79
> Cassandra-2.2.1 via apt-get
> 1x Intel Quad-Core Xeon E3-1230 / 16GB / 4x1TB SATA / 3x1TB RAID0 data drive
> Reporter: Ara Sadoyan
> Attachments: nodetool.txt, system.log
>
>
> Adding new node in heavy loaded environment freeze bootstrap. No errors are
> reported in log files. Some of other other nodes throws "String didn't
> validate" error, but I;m not sure that this is related.
> After restarting node it start bootstrap again and hangs after some time .
> nodetool netstats shows :
> /data/XXX/XXXX/tmp-la-1184-big-Data.db 5126078789/18345924701 bytes(27%)
> received from idx:0/192.168.220.16
> /data/XXX/XXXX/tmp-la-1233-big-Data.db 7213706459/18600941671 bytes(38%)
> received from idx:0/192.168.220.22
> /data/XXX/XXXX/tmp-la-1599-big-Data.db 8492408759/17572043398 bytes(48%)
> received from idx:0/192.168.220.12
> /data/XXX/XXXX/tmp-la-2066-big-Data.db 15773981555/18508127610 bytes(85%)
> received from idx:0/192.168.220.18
> /data/XXX/XXXX/tmp-la-211-big-Data.db 8274231066/17172754085 bytes(48%)
> received from idx:0/192.168.220.20
> but listing files on local FS shows "No such file or directory"
> This happens only if there is significant amount of data. I have 1.5 TB per
> node on 13 node cluster, we use STCS compaction strategy and flat network
> topology .
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
