On Tue, Oct 27, 2009 at 10:10 PM, Edmond Lau <[email protected]> wrote: > I'd like to improve my mental model of how Cassandra bootstrapping > works. My understanding is that bootstrapping is just an extra step > during a node's startup where the node copies data from neighboring > nodes that, according to its token, it should own; afterwards, the > node behaves like any other node. > > If that's correct, I have a few follow-up questions: > > - At what point does the new node get inserted into the hash ring so > that reads/writes for keys get directed to it?
It starts accepting reads when bootstrap is complete, but during bootstrap it will also be forwarded writes so it stays up-to-date. > - What are the semantics of bootstrapping a node that's been in the > cluster before and already has some data that's possibly outdated? > Should this work? This might be useful if a node's been out of > commission for sufficiently long period of time. Cluster membership is intended to be relatively stable. When removing nodes is supported (see https://issues.apache.org/jira/browse/CASSANDRA-435), it won't be worth special-casing the case of "new node, that has some data locally that is arbitrarily out of date." So you will want to wipe its data directories and re-bootstrap it. > - If we pick a poor set of initial tokens, would it be sensible to > modify the tokens on existing nodes and then restart them with > bootstrapping in order to rebalance? Not until https://issues.apache.org/jira/browse/CASSANDRA-193 is done so you can fix the damage you've done to your replicas. > I've also noticed that I can get my cassandra cluster into a weird > state via bootstrapping, where it stops accepting reads/writes. I'm > on Cassandra 0.4.1. A simple repro case is to start all 3 nodes of a > 3 node cluster (replication factor of 2) using bootstrapping. Getting > a key that I've inserted then leads to an IndexOutOfBoundsException. > Another IndexOutOfBoundsException was thrown later while flushing. Starting all nodes in bootstrap mode is not a supported operation. Don't do that. (Bootstrap is much more automatic in trunk, and fixes this specific problem as well as others.) -Jonathan
