I've no idea why it doesn't work well.
We are testing Elasticity of Cassandra 0.6.6.We choose
orderPreservingPartitioner and set replicationFactor as 2.
We start from 6-server cluster(node A, B,C,D,E,F), which is load balanced.
roughly every node has 12GB. we then add node G between A and B.
In general if you think the data is not distributed correctly run nodetool
repair on the node.
http://wiki.apache.org/cassandra/Operations#Repairing_missing_or_inconsistent_data
And before expecting the old node to throw away it's data, 'nodetool
cleanup' is required (but don't do this while
