It is also in the seeds list for the other nodes. Should I remove it from those, restart them one at a time, then restart it?
/etc/cassandra # grep -i bootstrap * doesn't show anything so I don't think I have auto_bootstrap false. Thanks very much for the help. On Mon, Apr 3, 2023 at 7:01 AM Carlos Diaz <crdiaz...@gmail.com> wrote: > Just remove it from the seed list in the cassandra.yaml file and restart > the node. Make sure that auto_bootstrap is set to true first though. > > On Sun, Apr 2, 2023 at 9:59 PM David Tinker <david.tin...@gmail.com> > wrote: > >> So likely because I made it a seed node when I added it to the cluster it >> didn't do the bootstrap process. How can I recover this? >> >> On Mon, Apr 3, 2023 at 6:41 AM David Tinker <david.tin...@gmail.com> >> wrote: >> >>> Yes replication factor is 3. >>> >>> I ran nodetool repair -pr on all the nodes (one at a time) and am still >>> having issues getting data back from queries. >>> >>> I did make the new node a seed node. >>> >>> Re "rack4": I assumed that was just an indication as to the physical >>> location of the server for redundancy. This one is separate from the others >>> so I used rack4. >>> >>> On Mon, Apr 3, 2023 at 6:30 AM Carlos Diaz <crdiaz...@gmail.com> wrote: >>> >>>> I'm assuming that your replication factor is 3. If that's the case, >>>> did you intentionally put this node in rack 4? Typically, you want to add >>>> nodes in multiples of your replication factor in order to keep the "racks" >>>> balanced. In other words, this node should have been added to rack 1, 2 or >>>> 3. >>>> >>>> Having said that, you should be able to easily fix your problem by >>>> running a nodetool repair -pr on the new node. >>>> >>>> On Sun, Apr 2, 2023 at 8:16 PM David Tinker <david.tin...@gmail.com> >>>> wrote: >>>> >>>>> Hi All >>>>> >>>>> I recently added a node to my 3 node Cassandra 4.0.5 cluster and now >>>>> many reads are not returning rows! What do I need to do to fix this? There >>>>> weren't any errors in the logs or other problems that I could see. I >>>>> expected the cluster to balance itself but this hasn't happened (yet?). >>>>> The >>>>> nodes are similar so I have num_tokens=256 for each. I am using the >>>>> Murmur3Partitioner. >>>>> >>>>> # nodetool status >>>>> Datacenter: dc1 >>>>> =============== >>>>> Status=Up/Down >>>>> |/ State=Normal/Leaving/Joining/Moving >>>>> -- Address Load Tokens Owns (effective) Host ID >>>>> Rack >>>>> UN xxx.xxx.xxx.105 2.65 TiB 256 72.9% >>>>> afd02287-3f88-4c6f-8b27-06f7a8192402 rack3 >>>>> UN xxx.xxx.xxx.253 2.6 TiB 256 73.9% >>>>> e1af72be-e5df-4c6b-a124-c7bc48c6602a rack2 >>>>> UN xxx.xxx.xxx.24 93.82 KiB 256 80.0% >>>>> c4e8b4a0-f014-45e6-afb4-648aad4f8500 rack4 >>>>> UN xxx.xxx.xxx.107 2.65 TiB 256 73.2% >>>>> ab72f017-be96-41d2-9bef-a551dec2c7b5 rack1 >>>>> >>>>> # nodetool netstats >>>>> Mode: NORMAL >>>>> Not sending any streams. >>>>> Read Repair Statistics: >>>>> Attempted: 0 >>>>> Mismatch (Blocking): 0 >>>>> Mismatch (Background): 0 >>>>> Pool Name Active Pending Completed Dropped >>>>> Large messages n/a 0 71754 0 >>>>> Small messages n/a 0 8398184 14 >>>>> Gossip messages n/a 0 1303634 0 >>>>> >>>>> # nodetool ring >>>>> Datacenter: dc1 >>>>> ========== >>>>> Address Rack Status State Load Owns >>>>> Token >>>>> >>>>> 9189523899826545641 >>>>> xxx.xxx.xxx.24 rack4 Up Normal 93.82 KiB >>>>> 79.95% -9194674091837769168 >>>>> xxx.xxx.xxx.107 rack1 Up Normal 2.65 TiB >>>>> 73.25% -9168781258594813088 >>>>> xxx.xxx.xxx.253 rack2 Up Normal 2.6 TiB >>>>> 73.92% -9163037340977721917 >>>>> xxx.xxx.xxx.105 rack3 Up Normal 2.65 TiB >>>>> 72.88% -9148860739730046229 >>>>> xxx.xxx.xxx.107 rack1 Up Normal 2.65 TiB >>>>> 73.25% -9125240034139323535 >>>>> xxx.xxx.xxx.253 rack2 Up Normal 2.6 TiB >>>>> 73.92% -9112518853051755414 >>>>> xxx.xxx.xxx.105 rack3 Up Normal 2.65 TiB >>>>> 72.88% -9100516173422432134 >>>>> ... >>>>> >>>>> This is causing a serious production issue. Please help if you can. >>>>> >>>>> Thanks >>>>> David >>>>> >>>>> >>>>> >>>>>
