For removing the bricks from the replica , we can just execute the
command "replace-brick" with "commit force" option
Following is the procedure to replace the brick in the replicated volume.
##Replacing brick in Replicate/Distributed Replicate volumes
This section of the document contains how brick:
`pranithk-laptop:/home/gfs/r2_0` is replaced with brick:
`pranithk-laptop:/home/gfs/r2_5` in volume `r2` with replica count `2`.
Steps:
0. Make sure there is no data in the new brick
pranithk-laptop:/home/gfs/r2_5
1. Check that all the bricks are running. It is okay if the brick that
is going to be replaced is down.
2. Bring the brick that is going to be replaced down if not already.
1. Get the pid of the brick by executing 'gluster volume <volname>
status'
```
12:37:49 ? gluster volume status
Status of volume: r2
Gluster process Port Online Pid
------------------------------------------------------------------------------
Brick pranithk-laptop:/home/gfs/r2_0 49152 Y 5342
Brick pranithk-laptop:/home/gfs/r2_1 49153 Y 5354
Brick pranithk-laptop:/home/gfs/r2_2 49154 Y 5365
Brick pranithk-laptop:/home/gfs/r2_3 49155 Y 5376
```
2. Login to the machine where the brick is running and kill the brick.
```
12:38:33 ? kill -9 5342
```
3. Confirm that the brick is not running anymore and the other bricks
are running fine.
```
12:38:38 ? gluster volume status
Status of volume: r2
Gluster process Port Online Pid
------------------------------------------------------------------------------
Brick pranithk-laptop:/home/gfs/r2_0 N/A N 5342
<<---- brick is not running, others are running fine.
Brick pranithk-laptop:/home/gfs/r2_1 49153 Y 5354
Brick pranithk-laptop:/home/gfs/r2_2 49154 Y 5365
Brick pranithk-laptop:/home/gfs/r2_3 49155 Y 5376
```
3. Using the gluster volume fuse mount (In this example: `/mnt/r2`) set
up metadata so that data will be synced to new brick (In this case it is
from `pranithk-laptop:/home/gfs/r2_1` to `pranithk-laptop:/home/gfs/r2_5`)
1. Create a directory on the mount point that doesn't already exist.
Then delete that directory, do the same for metadata changelog by doing
setfattr. This operation marks the pending changelog which will tell
self-heal damon/mounts to perform self-heal from /home/gfs/r2_1 to
/home/gfs/r2_5.
```
mkdir /mnt/r2/<name-of-nonexistent-dir>
rmdir /mnt/r2/<name-of-nonexistent-dir>
setfattr -n trusted.non-existent-key -v abc /mnt/r2
setfattr -x trusted.non-existent-key /mnt/r2
```
2. Check that there are pending xattrs:
```
getfattr -d -m. -e hex /home/gfs/r2_1
# file: home/gfs/r2_1
security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000
trusted.afr.r2-client-0=0x000000000000000300000002 <<---- xattrs
are marked from source brick pranithk-laptop:/home/gfs/r2_1
trusted.afr.r2-client-1=0x000000000000000000000000
trusted.gfid=0x00000000000000000000000000000001
trusted.glusterfs.dht=0x0000000100000000000000007ffffffe
trusted.glusterfs.volume-id=0xde822e25ebd049ea83bfaa3c4be2b440
```
4. Replace the brick with 'commit force' option. Please note that other
variants of replace-brick command are not supported.
1. Execute replace-brick command
```
12:58:46 ? gluster volume replace-brick r2
`hostname`:/home/gfs/r2_0 `hostname`:/home/gfs/r2_5 commit force
volume replace-brick: success: replace-brick commit successful
```
2. Check that the new brick is now online
```
12:59:21 ? gluster volume status
Status of volume: r2
Gluster process Port Online Pid
------------------------------------------------------------------------------
Brick pranithk-laptop:/home/gfs/r2_5 49156 Y 5731
<<<---- new brick is online
Brick pranithk-laptop:/home/gfs/r2_1 49153 Y 5354
Brick pranithk-laptop:/home/gfs/r2_2 49154 Y 5365
Brick pranithk-laptop:/home/gfs/r2_3 49155 Y 5376
```
3. Once self-heal completes the changelogs will be removed.
```
12:59:27 ? getfattr -d -m. -e hex /home/gfs/r2_1
getfattr: Removing leading '/' from absolute path names
# file: home/gfs/r2_1
security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000
trusted.afr.r2-client-0=0x000000000000000000000000 <<---- Pending
changelogs are cleared.
trusted.afr.r2-client-1=0x000000000000000000000000
trusted.gfid=0x00000000000000000000000000000001
trusted.glusterfs.dht=0x0000000100000000000000007ffffffe
trusted.glusterfs.volume-id=0xde822e25ebd049ea83bfaa3c4be2b440
```
On 08/27/2014 02:59 AM, Joseph Jozwik wrote:
To add to this it appears that replace brick is in a broken state. I
can't abort it, or commit it. And I can run any other commands until
it thinks the replace-brick is complete.
Is there a way to manually remove the task since it failed?
root@pixel-glusterfs1:/# gluster volume status gdata2tb
Status of volume: gdata2tb
Gluster process Port Online Pid
------------------------------------------------------------------------------
Brick 10.0.1.31:/mnt/data2tb/gbrick3 49157 Y 14783
Brick 10.0.1.152:/mnt/raid10/gbrick3 49158 Y 2622
Brick 10.0.1.153:/mnt/raid10/gbrick3 49153 Y 3034
NFS Server on localhost 2049 Y 14790
Self-heal Daemon on localhost N/A Y 14794
NFS Server on 10.0.0.205 N/A N N/A
Self-heal Daemon on 10.0.0.205 N/A Y 10323
NFS Server on 10.0.1.153 2049 Y 12735
Self-heal Daemon on 10.0.1.153 N/A Y 12742
NFS Server on 10.0.1.152 2049 Y 2629
Self-heal Daemon on 10.0.1.152 N/A Y 2636
Task ID Status
---- -- ------
Replace brick 1dace9f0-ba98-4db9-9124-c962e74cce07 completed
---------- Forwarded message ----------
From: *Joseph Jozwik* <[email protected]
<mailto:[email protected]>>
Date: Tue, Aug 26, 2014 at 3:42 PM
Subject: Moving brick of replica volume to new mount on filesystem.
To: [email protected] <mailto:[email protected]>
Hello,
I need to move a brick to another location on the filesystem.
My initial plan was to stop the gluster server with
1. service glusterfs-server stop
2. rsync -ap brick3 folder to new volume on server
3. umount old volume and bind mount the new to the same location.
However I stopped the glusterfs-server on the node and there was still
background processes running glusterd. So I was not sure how to safely
stop them.
I also attempted to replace-brick to a new location on the server but
that did not work with "volume replace-brick: failed: Commit failed on
localhost. Please check the log file for more details."
Then attempted remove brick with
"volume remove-brick gdata2tb replica 2 10.0.1.31:/mnt/data2tb/gbrick3
start"
gluster> volume remove-brick gdata2tb 10.0.1.31:/mnt/data2tb/gbrick3
status
volume remove-brick: failed: Volume gdata2tb is not a distribute
volume or contains only 1 brick.
Not performing rebalance
gluster>
Volume Name: gdata2tb
Type: Replicate
Volume ID: 6cbcb2fc-9fd7-467e-9561-bff1937e8492
Status: Started
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.0.1.31:/mnt/data2tb/gbrick3
Brick2: 10.0.1.152:/mnt/raid10/gbrick3
Brick3: 10.0.1.153:/mnt/raid10/gbrick3
_______________________________________________
Gluster-users mailing list
[email protected]
http://supercolony.gluster.org/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
[email protected]
http://supercolony.gluster.org/mailman/listinfo/gluster-users
