[
https://issues.apache.org/jira/browse/CASSANDRA-5915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13746604#comment-13746604
]
Brandon Williams commented on CASSANDRA-5915:
---------------------------------------------
I'm confused by how this worked at all the second time, because I encountered
CASSANDRA-5916 in all attempts. But I'm also confused how you can be missing
the STATUS state, since a failed replace won't cause this, it will just leave
the node with STATUS:hibernate,true. That said, I think the right path is "why
are things flapping so much" since bootstrap already waits for RING_DELAY for
things to stabilize, which at 30s should be plenty of time on any competent
network. (And it's worth noting you can override ring delay next time if you
need to wait longer.)
> node flapping prevents replace_node from succeeding consistently
> ----------------------------------------------------------------
>
> Key: CASSANDRA-5915
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5915
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: 1.2.8
> Reporter: Chris Burroughs
> Attachments: cassandra.log.gz
>
>
> A node was down for a week or two due to hardware disk failure. I tried to
> use replace_node to bring up a new node on the same physical host with the
> same IPs. (rbranson suspected that using the same IP may be more issue
> prone.) This failed due to "unable to find sufficient sources for streaming
> range" See CASSANDRA-5913 for a problem with how the failure was handled by
> gossip.
> All of the other nodes should have been up the entire time, but when this
> node came up it saw nodes flap up and down for quiet some time. I was
> eventually able to get replace_token to work by adding a 60 (!) second sleep
> to StorageService:bootstrap. I don't know if the right path is "why are
> things flapping so much" or "bootstrap should wait until things look stable".
> A few notes about the cluster:
> * 2 dc cluster (about 20 each), using GossipingPropertyFileSnitch
> * multi-dc no vpn setup:
> http://mail-archives.apache.org/mod_mbox/cassandra-user/201306.mbox/%3c51bf5c79.7020...@gmail.com%3E
> Startup log from the successful (with sleep) replace_node attached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
