On 03/25/2011 10:12 AM, Jonathan Ellis wrote:
On Fri, Mar 25, 2011 at 11:59 AM, ian douglas<i...@armorgames.com> wrote:
(we're running v0.60)
I don't know if you could hear that from where you are, but our whole
office just yelled, "WTF!" :)
Ah, that's what that noise was... And yeah, we know we're way behind. Our
initial delay in upgrading was waiting for 0.7 to come out and then we
learned we needed a whole new Thrift client for our PHP code base, and then
we got busy on other things, but we're at a point where we have some time to
take care of Cassandra and get it upgraded.
Our planned path, now, is:
(our nodes' tokens are numbered using the python code (0, 1/3 and 2/3 times
2^127), and called node 1 through 3, respectively; our RF is set to 2 right
now)
1. remove node 1 from our software
2. bring node 1 offline after a flush/repair/cleanup
3. run a cleanup on node 2 and then on node 3 so they have a full copy of
all data from the old node 1 and each other.
4. bring up a new Large 64-bit instance, install 0.6.12, assign a Token
value of 0 (node 1), RF:2, on a new gossip ring, and copy all data from the
32-bit nodes 2 and 3 and run a repair/cleanup to remove any duplicated data
5. remove node 3 from our software
6. point our code to the new 64-bit node 1
7. bring node 3 offline after a flush/repair/cleanup so node 2 has the last
fresh copy of everything
8. bring node 2 offline after a flush/repair/cleanup
9. bring up another Large instance, get a copy of all data from our old node
2, assign a Token value of (1/2 * 2^127), RF:2, on the new gossip ring, run
a repair to remove duplicate data, and then a cleanup so it gets replicated
data from the new node 1
10. add the new node 2 to our software
11. run a final cleanup on the new node 1 and then on node 2 to make sure
all data is replicated evenly on both nodes
... at this point, we should have two 64-bit Large instances, with RF:2, on
a new gossip ring, replacing three 32-bit systems, with minimal down time
and no data loss (just a data delay between steps 6 and 10 above).
Questions:
1. Does it appear that we've missed any steps, or doing something out of
order?
2. Is the flush/repair/cleanup overkill when bringing the old nodes offline,
or is that the correct sequence to follow?
3. Will the difference in compute units (lower on Large instances than
Medium instances) make any noticeable difference, or will the fact that the
machine is 64-bit handle things efficiently enough such that a Large
instance works harder than a Medium instance? (never did figure out their
how their compute units work)
4. Can we follow similar steps when we're ready to upgrade to 0.7x and have
our new Thrift client for PHP all squared away?
Thanks again for the help!!!
