Re: cassandra node stops streaming data during nodetool rebuild

2017-04-07 Thread Jacob Shadix
I don't see an issue with the size of the data / node. You can attempt the rebuild again and play around with throughput if your network can handle it. It can be changed on-the-fly with nodetool: nodetool setstreamthroughput This article is also worth a read -

Re: cassandra node stops streaming data during nodetool rebuild

2017-04-07 Thread Roland Otta
good point! on the source side i can see the following error ERROR [STREAM-OUT-/192.168.0.114:34094] 2017-04-06 17:18:56,532 StreamSession.java:529 - [Stream #41606030-1ad9-11e7-9f16-51230e2be4e9] Streaming error occurred on session with peer 10.192.116.1 through 192.168. 0.114

Re: cassandra node stops streaming data during nodetool rebuild

2017-04-07 Thread Jacob Shadix
Did you look at the logs on the source DC as well? How big is the dataset? -- Jacob Shadix On Fri, Apr 7, 2017 at 7:16 AM, Roland Otta wrote: > Hi! > > we are on 3.7. > > we have some debug messages ... but i guess they are not related to that > issue > DEBUG

Re: cassandra node stops streaming data during nodetool rebuild

2017-04-07 Thread Roland Otta
Hi! we are on 3.7. we have some debug messages ... but i guess they are not related to that issue DEBUG [GossipStage:1] 2017-04-07 13:11:00,440 FailureDetector.java:456 - Ignoring interval time of 2002469610 for /192.168.0.27 DEBUG [GossipStage:1] 2017-04-07 13:11:00,441

Re: cassandra node stops streaming data during nodetool rebuild

2017-04-07 Thread Jacob Shadix
What version are you running? Do you see any errors in the system.log (SocketTimeout, for instance)? And what values do you have for the following in cassandra.yaml: - - stream_throughput_outbound_megabits_per_sec - - compaction_throughput_mb_per_sec - - streaming_socket_timeout_in_ms -- Jacob

cassandra node stops streaming data during nodetool rebuild

2017-04-07 Thread Roland Otta
hi, we are trying to setup a new datacenter and are initalizing the data with nodetool rebuild. after some hours it seems that the node stopped streaming (at least there is no more streaming traffic on the network interface). nodetool netstats shows that the streaming is still in progress