Well, that looks like your problem. They are logical racks and they come into play when NetworkTopologyStrategy is deciding which replicas to put data on. NTS will ensure a replica goes on the first node in a different rack when traversing the ring, with the idea of keeping only one set of replicas on a rack (so that a whole rack can go down without you losing QUORUM).
To re-balance it the simplest method will be to recommission node7 or 8 (probably 8) in a different rack and ensure you always have 1 rack per node. This may not be a good long term plan, so re-organising your cluster to use a different number of racks may be ideal (typically #racks == RF works best if possible), but that requires a far more elaborate and costly process to achieve which may not be suitable at the moment. Hypothetically speaking, how would I obtain "the tokens that will > balance the ring"? Are there tools out there? The token allocation algorithm is built into Cassandra, and can be set up for when you configure a new node by specifying the allocate_tokens_for_keyspace setting in the Cassandra yaml. In this case it will not help you because of the above mentioned rack situation. You should be fine once you solve the rack problem. raft.so - Cassandra consulting, support, and managed services On Thu, Mar 4, 2021 at 1:14 AM Lapo Luchini <l...@lapo.it> wrote: > Hi! The nodes are all in different racks… except for node7 and node8! > Which is something more that makes them similar (which I didn't notice > at first), other than timeline of being added to the cluster. > > About the token ring calculation… I'll retry that in NodeJS instead of > awk as a double check… yes, it checks out: > > 0.14603899130502265 node1 HEL1-DC2 > 0.14885298279986256 node2 FSN1-DC6 > 0.13917538352395356 node3 FSN1-DC12 > 0.13593194981676893 node4 FSN1-DC10 > 0.14054248949667470 node5 FSN1-DC11 > 0.14387515909570683 node7 FSN1-DC7 > 0.14558304396201086 node8 FSN1-DC7 > > (rack names are actually Hetzner datacenters, which is as accurate info > as I know… maybe those two FSN1-DC7 nodes are in different racks too) > > Yeah, I'm willing to "live with it", my question was more about on the > line of "is there something I didn't understand well in Cassandra?" than > being preoccupied about the difference in usage. > > All the involved keyspaces have an identical RF of: > {'class': 'NetworkTopologyStrategy', 'Hetzner': 3, 'DR': 1} > > Hypothetically speaking, how would I obtain "the tokens that will > balance the ring"? Are there tools out there? > > thanks in advance! > cheers, > Lapo > > On 2021-03-03 11:41, Kane Wilson wrote: > > The load calculation always has issues so I wouldn't count on it, > > although in this case it does seem to roughly line up. Are you sure your > > ring calculation was accurate? It doesn't really seem to line up with > > the owns % for the 33% node, and it is feasible (although unlikely) that > > you could roll a node with a bunch of useless tokens and end up in this > > scenario. > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org > For additional commands, e-mail: user-h...@cassandra.apache.org > >