I've been thinking a bit about the problem of fat nodes (say, 20gb per node). My understanding is that the amount of overhead of adding a new node into the ring is massive with fat nodes due to the fact that you have to stream in 20TB which takes forever.
In a scenario where a given node only has a single token, my assumption is this forces Cassandra to send the entire token range to the new node. As a potential approach, would it be possible for a node to incrementally acquire tokens, and as a result incrementally stream? You could have a node serving requests after acquiring 1 token, and it would gradually take ownership of more and more of the ring as it bootstraps. Thoughts? Jon
