Hi, I dont have any more hosts available.
I am a bit lost here, why a replica 3 and arbiter 1? ie not replica2 arbiter1? also no distributed part? is the distributed flag automatically assumed? with a replica3 then there is a quorum (2 of 3) so no arbiter is needed? I have this running already like this so I am assuming its robust? I am still struggling to undersatnd the syntax, I wish the docs / examples were better. So on each gluster node I have an-unused 120gb data1 partition which is left over from the OS install so the arbiter volume could go here? in which case? gluster volume create my-volume replica 2 arbiter 1 host1:/path/to/brick host2:/path/to/brick (arb-)host3:/path/to/brick2 host4:/path/to/brick host5:/path/to/brick (arb-)host6:/path/to/brick2 host3:/path/to/brick host6:/path/to/brick (arb-)host1:/path/to/brick2 is this a sane command? Otherwise maybe I am beginning to think I am better off doing 3 x 2TB separate volumes. rather interesting trying to understand this stuff...! On 12 June 2018 at 23:10, Dave Sherohman <d...@sherohman.org> wrote: > On Tue, Jun 12, 2018 at 03:04:14PM +1200, Thing wrote: > > What I would like to do I think is a, > > > > *Distributed-Replicated volume* > > > > a) have 1 and 2 as raid1 > > b) have 4 and 5 as raid1 > > c) have 3 and 6 as a raid1 > > d) join this as concatenation 2+2+2tb > > You probably don't actually want to do that because quorum is handled > separately for each subvolume (bricks 1/2, 4/5, or 3/6), not a single > quorum for the volume as a whole. (Consider if bricks 1 and 2 both went > down. You'd still have 4 of 6 bricks running, so whole-volume quorum > would still be met, but the volume can't continue to run normally since > the first subvolume is completely missing.) > > In the specific case of replica 2, gluster treats the first brick in > each subvolume as "slightly more than one", so you'd be able to continue > normally if brick 2, 5, or 6 went down, but, if brick 1, 4, or 3 went > down, all files on that subvolume would become read-only. > > > I tried to do this and failed as it kept asking for an arbiter, which the > > docs simply dont mention how to do. > > https://gluster.readthedocs.io/en/latest/Administrator% > 20Guide/arbiter-volumes-and-quorum/ > > > So say we have, > > > > a) have 1 and 2 as raid1 with 3 as the arbiter? > > b) have 4 and 5 as raid 1 with 6 as the arbiter > > c) 3 and 6 as a raid 1 with 5 as the arbiter > > d) join this as concatenation 2+2+2tb > > I would recommend finding one or more other servers with small amounts > of unused space and allocating the arbiter bricks there, or carving a > gig or two out of your current bricks for that purpose. Arbiters only > need about 4k of disk space per file in the subvolume, regardless of the > actual file size (the arbiter only stores metadata), so TB-sized > arbiters would be a huge waste of space, especially if you're only > putting a few very large files (such as VM disk images) on the volume. > > As a real-world data point, I'm using basically the setup you're aiming > for - six data bricks plus three arbiters, used to store VM disk images. > My data bricks are 11T each, while my arbiters are 98G. Disk usage for > the volume is currently at 19%, but all arbiters are under 1% usage (the > largest has 370M used). Assuming my usage patterns don't change, I > could completely fill my 11T subvolumes and only need about 1.5G in the > corresponding arbiters. > > > if so what is the command used to build this? > > # gluster volume create my-volume replica 3 arbiter 1 host1:/path/to/brick > host2:/path/to/brick arb-host1:/path/to/brick host4:/path/to/brick > host5:/path/to/brick arb-host2:/path/to/brick host3:/path/to/brick > host6:/path/to/brick arb-host3:/path/to/brick > > -- > Dave Sherohman > _______________________________________________ > Gluster-users mailing list > Gluster-users@gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-users >
_______________________________________________ Gluster-users mailing list Gluster-users@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-users