On 9/9/2014 7:12 PM, Michael Bushey wrote:
I'm currently using GlusterFS 3.5.2 on a pair of production servers to
share uploaded files and it's been reliable and working well with just the
two servers. I've done some local tests of trying to add and remove servers
and the results have not been good. I'm starting to think I have the wrong
idea of what GlusterFS does and I'm trying to stick the square peg in the
round hole. Is there some config to say "number of replicas = number of
servers"?
My need is to have a copy of the data local on each server. If I have 5
servers, I want five copies of the data. Basically each server should have
it's own copy like it's a local ext4/zfs file-system, except it will
immediately sync files added or removed on the other servers. I'm not sure
the what the gluster definition of brick is, but I believe I want the
number of bricks to equal the number of servers (at least for this
particular file-system).
I've played around a bit and lost my data every time I've tried to add or
remove a node. There is plenty of documentation, but it all says the same
thing and doesn't answer the basics.
From `gluster volume info`: Number of Bricks: 1 x 2 = 2
I'm assuming the middle 2 is the number of bricks. I have no clue why we're
multiplying by 1 to get itself.
http://gluster.org/community/documentation/index.php/Gluster_3.2:_Displaying_Volume_Information
- does not show this magic multiplication. The page for 3.3 does not exist.
We're cloud based and we want to be able to add and remove servers on the
fly. If for example I want to scale from 5 to 7 servers, it looks like I
need to create a new storage pool with 7 replicas. Would it be the most
logical to name my brick with the number of replicas? For example server1
to server5 already have /var/gluster/files5. I could then create
/var/gluster/files7 on server1 to server7, create the pool with 7 replicas,
then pick a machine like server1 to copy the data from /var/gluster/files5
to /var/gluster/files7, then destroy /var/gluster/files5. This seems
extremely convoluted but it does not appear possible to expand a storage
pool and expand the number of replicants with it.
Thanks in advance for help and your time to read this.
Michael
I have done this a few times, but it is not real recent, so I will try to
explain in a way that is both correct and understandable.
1. As to the question about the "`gluster volume info`: Number of Bricks: 1 x
2 = 2"
The first number is the number of bricks that the data is distributed
across. If that number is 3, each of three bricks has 1/3 of the data. This
is the "Distribute" mode of operating gluster. From what you describe, you
are not interested in that. You are interested in the "Replicate" mode. The
numbering was adopted because gluster makes it possible to use various
combinations of Distribute and Replicate at the same time, and this numbering
describes the arrangement very succinctly.
2. As to how to you add a brick to your volume, so you can add a server to
your system. You use the 'gluster volume add-brick' command. You are
telling gluster: You are currently a two-brick system and I want to make you
a 3-brick system (each brick being on a different server) by adding
'server3'. Typing 'gluster volume add-brick help' returns
gluster volume add-brick <VOLNAME> [<stripe|replica> <COUNT>] <NEW-BRICK> ...
[force]
Assuming that the brick is ready and freshly formatted with xfs, the command
ends up looking something like:
gluster volume add-brick upload replica 3 server3:/bricks/upload/upload
This will create the gluster structure on the brick, and will start the
process of replicating the gluster data onto the brick. The gluster mount
should be usable at that point, pulling the data from other bricks if it has
not been replicated onto the local brick yet.
If this server has been in service before, it is probably best to reformat
the brick before putting it into service. Otherwise gluster will see
remnants of the past use on the brick, and not want to use it, assuming there
may be valuable information already stored there.
CAUTION: Unless they have fixed this issue, the replication process can bring
a server to its knees until it is fully replicated.
3. To remove a server from your system, the process is basically reversed.
If you want to go from 5 servers to 4, you issue the command:
gluster volume remove-brick upload replica 4 server5:/bricks/upload/upload
This tells gluster to drop the brick on server5 from the volume, and
reassures gluster that you know that you will now have a replicated volume
with 4 replicas on it.
Hope this helps.
Ted Miller
Elkhart, IN
P.S. I am not reading the list regularly, so if you need more info, best to
copy me when sending to the list. Just saw your request and said "I've done
that, I can share some information."
_______________________________________________
Gluster-users mailing list
[email protected]
http://supercolony.gluster.org/mailman/listinfo/gluster-users
