Hello,
Nine days ago, I bootstrapped an Cassandra node, let's call it
10.1.2.11. The node is part of a six-node Cassandra 2.1.11 cluster
spread across two datacenters. The bootstrapping was conducted the
following way on the node:
1. Shut down Cassandra.
2. Deleted its data.
3. Temporarily adjusted its configuration to be in bootstrapping node:
- In cassandra-env.sh:
- JVM_OPTS="$JVM_OPTS -Dcassandra.replace_address=ipaddressofnode"
- In cassandra.yaml:
- auto_bootstrap: true
- Made sure the nodes' IP address was not listed in seed_provider
4. Started Cassandra again.
That seemed to go well: In system.log the following was seen:
INFO [main] 2019-02-08 14:49:17,223 StorageService.java:1120 - JOINING:
Starting to bootstrap...
No errors seen in system.log.
A graph of the server's data filesystem shows fine growth during the
next ~24 hours.
However, it seems the joining is stuck without completing.
Parts of output from "nodetool netstats", note the "JOINING" mode:
================================================================
[cas@nodename ~]$ nodetool netstats
Mode: JOINING
Bootstrap 53d1a8a0-2ba8-11e9-9212-2d89290b6701
/10.1.2.10
/10.1.1.10
/10.1.1.12
/10.1.2.12
Receiving 225 files, 461265115573 bytes total. Already received
225 files, 461265115573 bytes total
/data/cassandra/ksname/...db 43392372/43392372 bytes(100%)
received from idx:0/10.1.2.12
[... lots of lines like this ...]
/data/cassandra/ksname/...db 131825151/131825151 bytes(100%)
received from idx:0/10.1.2.12
/10.1.1.11
Read Repair Statistics:
Attempted: 0
Mismatch (Blocking): 0
Mismatch (Background): 0
Pool Name Active Pending Completed
Commands n/a 0 5
Responses n/a 0 3153171
================================================================
All the "...received from..." lines show 100% bytes received.
Parts of output from "nodetool status":
================================================================
[cas@nodename ~]$ nodetool status
Datacenter: DC1
===============
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID Rack
UN 10.1.1.12 454.14 GB 256 ? c100e77d-... RAC1
UN 10.1.1.11 449.99 GB 256 ? 0e06dc69-... RAC1
UN 10.1.1.10 725.81 GB 256 ? f266489b-... RAC1
Datacenter: DC2
===============
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID Rack
UN 10.1.2.12 474.59 GB 256 ? 89b71027-... RAC1
UN 10.1.2.11 455.1 GB 256 ? c08b8e99-... RAC1
UN 10.1.2.10 475 GB 256 ? e17bae55-... RAC1
================================================================
How come "nodetool netstats" is still showing "Mode: JOINING"? Is there
a way I can push the node to complete its joining?
--
Kind regards,
Troels Arvin
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org
