On 08/13/2013 03:43 AM, Cool wrote:
remove-brick in 3.4.0 seems removing wrong bricks, can someone help to
review the environment/steps to see if I did anything stupid?
setup - Ubuntu 12.04LTS on gfs11 and gfs12, with following packages
from ppa, both nodes have 3 xfs partitions sdb1, sdc1, sdd1:
ii glusterfs-client 3.4.0final-ubuntu1~precise1
clustered file-system (client package)
ii glusterfs-common 3.4.0final-ubuntu1~precise1
GlusterFS common libraries and translator modules
ii glusterfs-server 3.4.0final-ubuntu1~precise1
clustered file-system (server package)
step to reproduce the problem:
1. create volume gfs_v0 in replica 2 with gfs11:/sdb1 and gfs12:/sdb1
2. add-brick gfs11:/sdc1 and gfs12:/sdc1
3. add-brick gfs11:/sdd1 and gfs12:/sdd1
4. rebalance to make files distributed to all three pair of disks
5. remove-brick gfs11:/sdd1 and gfs12:/sdd1 start, files on
***/sdc1*** are migrating out
6. remove-brick commit led to data loss in gfs_v0
If between step 5 and 6 I initiate a remove-brick targeting /sdc1,
then after commit I would not lose anything since all data will be
migrated back to /sdb1.
You should ensure that a 'remove-brick start ' has completed and then
commit it before initiating the second one. The correct way to do this
would be:
5. # gluster volume remove-brick gfs_v0 gfs11:/sdd1 gfs12:/sdd1 start
6. Check that the data migration has been completed using the status
command:
# gluster volume remove-brick gfs_v0 gfs11:/sdd1 gfs12:/sdd1 status
7. #gluster volume remove-brick gfs_v0 gfs11:/sdd1 gfs12:/sdd1 commit
8. # gluster volume remove-brick gfs_v0 gfs11:/sdc1 gfs12:/sdc1 start
9. # gluster volume remove-brick gfs_v0 gfs11:/sdc1 gfs12:/sdc1 status
10. # gluster volume remove-brick gfs_v0 gfs11:/sdc1 gfs12:/sdc1 commit
This would leave you with the original replica 2 volume that you had
begun with. Hope this helps.
Note:
The latest version of glusterfs has the check that prevents a second
remove-brick operation until the first one has been committed.
(You would receive a message thus : "volume remove-brick start: failed:
An earlier remove-brick task exists for volume <volname>. Either commit
it or stop it before starting a new task." )
-Ravi
-C.B.
_______________________________________________
Gluster-users mailing list
[email protected]
http://supercolony.gluster.org/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
[email protected]
http://supercolony.gluster.org/mailman/listinfo/gluster-users
