Thanks guys. I have to replace all data disks, so I don't have another large enough local disk to move the data to. If I'll have no choice, I will backup the data before on some other node or something, but I'd like to avoid it. I would really love letting Cassandra do it thing and rebuild itself. Did anybody handled such cases that way (Letting Cassandra rebuild it's data?) Although there are no documented procedure for it, It should be possible right?
On Fri, Dec 19, 2014 at 8:41 AM, Jan Kesten <j.kes...@enercast.de> wrote: > Hi Or, > > I did some sort of this a while ago. If your machines do have a free disk > slot - just put another disk there and use it as another > data_file_directory. > > If not - as in my case: > > - grab an usb dock for disks > - put the new one in there, plug in, format, mount to /mnt etc. > - I did an online rsync from /var/lib/cassandra/data to /mnt > - after that, bring cassandra down > - do another rsync from /var/lib/cassandra/data to /mnt (should be faster, > as sstables do not change, minimizes downtime) > - if you need adjust /etc/fstab if needed > - shutdown the node > - swap disks > - power on the node > - everything should be fine ;-) > > Of course you will need a replication factor > 1 for this to work ;-) > > Just my 2 cents, > Jan > > rsync the full contents there, > > Am 18.12.2014 um 16:17 schrieb Or Sher: > > Hi all, >> >> We have a situation where some of our nodes have smaller disks and we >> would like to align all nodes by replacing the smaller disks to bigger ones >> without replacing nodes. >> We don't have enough space to put data on / disk and copy it back to the >> bigger disks so we would like to rebuild the nodes data from other replicas. >> >> What do you think should be the procedure here? >> >> I'm guessing it should be something like this but I'm pretty sure it's >> not enough. >> 1. shutdown C* node and server. >> 2. replace disks + create the same vg lv etc. >> 3. start C* (Normally?) >> 4. nodetool repair/rebuild? >> *I think I might get some consistency issues for use cases relying on >> Quorum reads and writes for strong consistency. >> What do you say? >> >> Another question is (and I know it depends on many factors but I'd like >> to hear an experienced estimation): How much time would take to rebuild a >> 250G data node? >> >> Thanks in advance, >> Or. >> >> -- >> Or Sher >> > > -- Or Sher
