[
https://issues.apache.org/jira/browse/CASSANDRA-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15064093#comment-15064093
]
Stefania commented on CASSANDRA-8072:
-------------------------------------
I've reproduced it on 2.1 HEAD with this dtest, which follows the steps of
CASSANDRA-8422:
{code}
def decommissioned_wiped_node_can_gossip_to_single_seed_test(self):
"""
@jira_ticket CASSANDRA-8072
@jira_ticket CASSANDRA-8422
Test that if we decommission a node, kill it and wipe its data, it can
join a cluster with a single
seed node.
"""
cluster = self.cluster
cluster.populate(1)
cluster.start(wait_for_binary_proto=True)
# Add a new node, bootstrap=True ensures that it is not a seed
node2 = new_node(cluster, bootstrap=True)
node2.start(wait_for_binary_proto=True, wait_other_notice=True)
# Decommision the new node and kill it
node2.decommission()
node2.stop(gently=False)
# Wipe its data
data_dir = os.path.join(node2.get_path(), 'data')
commitlog_dir = os.path.join(node2.get_path(), 'commitlogs')
debug("Deleting {}".format(data_dir))
shutil.rmtree(data_dir)
shutil.rmtree(commitlog_dir)
# Now start it, it should be allowed to join
mark = node2.mark_log()
node2.start(wait_other_notice=False)
node2.watch_log_for("JOINING:", from_mark=mark)
{code}
This results in the following exception in node2:
{code}
ERROR [main] 2015-12-18 16:21:07,357 CassandraDaemon.java:581 - Exception
encountered during startup
java.lang.RuntimeException: Unable to gossip with any seeds
at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1337)
~[main/:na]
at
org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:541)
~[main/:na]
at
org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:789)
~[main/:na]
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:721)
~[main/:na]
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:612)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:389)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:564)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:653)
[main/:na]
WARN [StorageServiceShutdownHook] 2015-12-18 16:21:07,360 Gossiper.java:1454 -
No local state or state is in silent shutdown, not announcing shutdown
INFO [StorageServiceShutdownHook] 2015-12-18 16:21:07,361
MessagingService.java:734 - Waiting for messaging service to quiesce
INFO [ACCEPT-/127.0.0.2] 2015-12-18 16:21:07,361 MessagingService.java:1018 -
MessagingService has terminated the accept() thread
{code}
> Exception during startup: Unable to gossip with any seeds
> ---------------------------------------------------------
>
> Key: CASSANDRA-8072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8072
> Project: Cassandra
> Issue Type: Bug
> Components: Lifecycle
> Reporter: Ryan Springer
> Assignee: Stefania
> Fix For: 2.1.x
>
> Attachments: cas-dev-dt-01-uw1-cassandra-seed01_logs.tar.bz2,
> cas-dev-dt-01-uw1-cassandra-seed02_logs.tar.bz2,
> cas-dev-dt-01-uw1-cassandra02_logs.tar.bz2,
> casandra-system-log-with-assert-patch.log, screenshot-1.png,
> trace_logs.tar.bz2
>
>
> When Opscenter 4.1.4 or 5.0.1 tries to provision a 2-node DSC 2.0.10 cluster
> in either ec2 or locally, an error occurs sometimes with one of the nodes
> refusing to start C*. The error in the /var/log/cassandra/system.log is:
> ERROR [main] 2014-10-06 15:54:52,292 CassandraDaemon.java (line 513)
> Exception encountered during startup
> java.lang.RuntimeException: Unable to gossip with any seeds
> at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1200)
> at
> org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:444)
> at
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:655)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:609)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:502)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585)
> INFO [StorageServiceShutdownHook] 2014-10-06 15:54:52,326 Gossiper.java
> (line 1279) Announcing shutdown
> INFO [StorageServiceShutdownHook] 2014-10-06 15:54:54,326
> MessagingService.java (line 701) Waiting for messaging service to quiesce
> INFO [ACCEPT-localhost/127.0.0.1] 2014-10-06 15:54:54,327
> MessagingService.java (line 941) MessagingService has terminated the accept()
> thread
> This errors does not always occur when provisioning a 2-node cluster, but
> probably around half of the time on only one of the nodes. I haven't been
> able to reproduce this error with DSC 2.0.9, and there have been no code or
> definition file changes in Opscenter.
> I can reproduce locally with the above steps. I'm happy to test any proposed
> fixes since I'm the only person able to reproduce reliably so far.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
