> On Jan 31, 2018, at 12:35 AM, Oleksandr Shulgin
> <oleksandr.shul...@zalando.de> wrote:
>
>> On Tue, Jan 30, 2018 at 5:44 PM, Jeff Jirsa <jji...@gmail.com> wrote:
>> All DCs in a cluster use the same token space in the DHT,
>
> I can't believe my bloody eyes, but this seems to be true...
I don’t know why this is a surprise (maybe because people like to talk about
multiple rings, but the fact that replication strategy is set per keyspace and
that you could use SimpleStrategy in a multiple dc cluster demonstrates this),
but we can chat about that another time
>
>> so token conflicts across datacenters are invalid config
>
> If this is deemed invalid config why does the new node *silently* steals the
> existing token, badly affecting the ownership of the rest of the nodes? It
> should just refuse to start!
Philosophically, With multiple DCs, it may start up and not see the other DC
for minutes/hours/days before it realizes there’s a token conflict - what
should it do then? Which node gets stopped? If your suggestion to resolve that
is to make sure we see the whole ring before starting up, we end up in a
situation where we try not to startup unless we can see all nodes, and create
outages during DC separations.
Distributed systems and occasional availability make these decisions harder.
Please open a jira if you think it’s wrong, but I’m not sure I know what the
“right” answer is either.
