Likely lost (enough of) the system key space on that disk so the data files indicating the host was in the cluster are missing and the host tried to rebootstrap
> On Dec 11, 2021, at 12:47 PM, Bowen Song <bo...@bso.ng> wrote: > > > Hi Joss, > > To unsubscribe from this mailing list, please send an email to > user-unsubscr...@cassandra.apache.org, not the mailing list itself > (user@cassandra.apache.org). > >> On 09/12/2021 16:14, Joss wrote: >> unsubscribe >> >> On Mon, 6 Dec 2021 at 14:12, Joe Obernberger <joseph.obernber...@gmail.com> >> wrote: >>> Hi All - one node in an 11 node cluster experienced a drive failure on >>> the first drive in the list. I removed that drive from the list so that >>> it now reads: >>> >>> data_file_directories: >>> - /data/2/cassandra/data >>> - /data/3/cassandra/data >>> - /data/4/cassandra/data >>> - /data/5/cassandra/data >>> - /data/6/cassandra/data >>> - /data/8/cassandra/data >>> - /data/9/cassandra/data >>> >>> But when I try to start the server, I get: >>> >>> Exception (java.lang.RuntimeException) encountered during startup: A >>> node with address /172.16.100.251:7000 already exists, cancelling join. >>> Use cassandra.replace_address if you want to replace this node. >>> java.lang.RuntimeException: A node with address /172.16.100.251:7000 >>> already exists, cancelling join. Use cassandra.replace_address if you >>> want to replace this node. >>> at >>> org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:659) >>> at >>> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:934) >>> at >>> org.apache.cassandra.service.StorageService.initServer(StorageService.java:784) >>> at >>> org.apache.cassandra.service.StorageService.initServer(StorageService.java:729) >>> at >>> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:420) >>> at >>> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:763) >>> at >>> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:887) >>> ERROR [main] 2021-12-05 15:49:48,446 CassandraDaemon.java:909 - >>> Exception encountered during startup >>> java.lang.RuntimeException: A node with address /172.16.100.251:7000 >>> already exists, cancelling join. Use cassandra.replace_address if you >>> want to replace this node. >>> at >>> org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:659) >>> at >>> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:934) >>> at >>> org.apache.cassandra.service.StorageService.initServer(StorageService.java:784) >>> at >>> org.apache.cassandra.service.StorageService.initServer(StorageService.java:729) >>> at >>> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:420) >>> at >>> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:763) >>> at >>> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:887) >>> INFO [StorageServiceShutdownHook] 2021-12-05 15:49:48,468 >>> HintsService.java:220 - Paused hints dispatch >>> WARN [StorageServiceShutdownHook] 2021-12-05 15:49:48,470 >>> Gossiper.java:1993 - No local state, state is in silent shutdown, or >>> node hasn't joined, not announcing shutdown >>> >>> Do I need to remove and re-add the node? When a drive fails with >>> cassandra, is it common for the node to come down? >>> >>> Thank you! >>> >>> -Joe Obernberger >>>
