I have a simple setup:

gluster> volume info

Volume Name: myvolume
Type: Distributed-Replicate
Status: Started
Number of Bricks: 3 x 2 = 6
Transport-type: tcp
Bricks:
Brick1: 10.2.218.188:/srv
Brick2: 10.116.245.136:/srv
Brick3: 10.206.38.103:/srv
Brick4: 10.114.41.53:/srv
Brick5: 10.68.73.41:/srv
Brick6: 10.204.129.91:/srv

I *killed* Brick #4 (kill -9 and then shut down instance).

My intention is to simulate a catastrophic failure of Brick4 and replace it
with a new server.

I probed the new server, then ran the following command:

gluster> peer probe 10.76.242.97
Probe successful

gluster> volume replace-brick myvolume 10.114.41.53:/srv 10.76.242.97:/srv
start
replace-brick started successfully

I waited a little while, saw no traffic on the new server and then ran this:

gluster> volume replace-brick myvolume 10.114.41.53:/srv 10.76.242.97:/srv
status

It never returned.  Now my cluster is in some weird state.  It's still
serving files, I still have a job copying files to it, but I am unable to
replace the bad peer with a new one.

root@ip-10-2-218-188:~# gluster volume replace-brick myvolume 10.114.41.53:/srv
10.76.242.97:/srv status
replace-brick status unknown

root@ip-10-2-218-188:~# gluster volume replace-brick myvolume 10.114.41.53:/srv
10.76.242.97:/srv abort
replace-brick abort failed

root@ip-10-2-218-188:~# gluster volume replace-brick myvolume 10.114.41.53:/srv
10.76.242.97:/srv start
replace-brick failed to start

How can I get my cluster back into a clean working state?

Thanks,
Bryan
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users

Reply via email to