Re: [ovirt-users] why replica 3
On 09/09/2015 10:54 AM, Simone Tiraboschi wrote: > > > On Wed, Sep 9, 2015 at 10:14 AM, Richard Neuboeck > mailto:h...@tbi.univie.ac.at>> wrote: > > On 04.09.15 10:02, Simone Tiraboschi wrote: > > Is there a reason why it has to be exactly replica 3? > > > > > > To have a valid quorum having the system being able to decide witch is > > the right and safe copy avoiding an issue called split brain. > > Under certain circumstances/issues (network issue, hosts down or > > whatever could happen) the data on different replica could diverge: if > > you have two and just two different hosts that claim each other > that its > > copy is the right one there is no way to automatically take the right > > decision. Having three hosts and setting the quorum according to that > > solves/mitigates the issue. > > > Thanks for the explanation. I do understand the problem but since > I'm somewhat limited in my hardware options is there a way to > override this requirement? Meaning if I change the checks for > replica 3 in the installation scripts does something else fail on > the way? > > > I'm advising that it's not a safe configuration so it's not > recommended for a production environment. > Having said that, as far as I know it's enforced only in the setup > script so tweaking it should be enough. > Otherwise, if you have enough disk space, you can also have a > different trick: you could create a replica 3 volume with 2 bricks > from a single host. I've thought about that but since that would obviously only help to fool the installation script there is nothing else in this setup that would improve the situation. Worse the read/write overhead on the second machine would be a performance downgrade. > It's not a safe procedure at all cause you still have only 2 hosts, > so it's basically just replica 2, and in case of split brain the > host with two copies will win by configuration which is not always > the right decision. Right. I'm thinking of trying to add a dummy node as mentioned in the RHEL documentation. This would (in theory) prevent the read only state in the split brain scenario and make it possible to access the storage. But still the installation requirement of replica 3 would not be satisfied. > > In my case coherence checks would come from outside the storage and > vm host setup and fencing would be applied appropriately. > > > Can I ask how? Multiple machines separated from the storage and virtualization machines that will check communication (in general and of several services) and try to intervene if there is something going awry first by accessing the machines directly (if possible) and then by deactivating those machines by remote management. Cheers Richard > > I would very much appreciate it if the particulars of the storage > setup could be either selected from a list of possibilities or be > ignored and just a warning be issued that this setup is not > recommended. > > Thanks! > Richard > > > -- > /dev/null > > -- /dev/null signature.asc Description: OpenPGP digital signature ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] why replica 3
On Wed, Sep 9, 2015 at 10:49 AM, Jorick Astrego wrote: > > > On 09/09/2015 10:14 AM, Richard Neuboeck wrote: > > On 04.09.15 10:02, Simone Tiraboschi wrote: > >> Is there a reason why it has to be exactly replica 3? > >> > >> > >> To have a valid quorum having the system being able to decide witch is > >> the right and safe copy avoiding an issue called split brain. > >> Under certain circumstances/issues (network issue, hosts down or > >> whatever could happen) the data on different replica could diverge: if > >> you have two and just two different hosts that claim each other > > that its > >> copy is the right one there is no way to automatically take the right > >> decision. Having three hosts and setting the quorum according to that > >> solves/mitigates the issue. > > > > Thanks for the explanation. I do understand the problem but since > > I'm somewhat limited in my hardware options is there a way to > > override this requirement? Meaning if I change the checks for > > replica 3 in the installation scripts does something else fail on > > the way? > > > > In my case coherence checks would come from outside the storage and > > vm host setup and fencing would be applied appropriately. > > > > I would very much appreciate it if the particulars of the storage > > setup could be either selected from a list of possibilities or be > > ignored and just a warning be issued that this setup is not recommended. > > > > Thanks! > > Richard > > > > > As a side question, in Glusterfs 3.7 there is an "AFR arbiter volume" > ( > http://gluster.readthedocs.org/en/release-3.7.0-1/Features/afr-arbiter-volumes/) > > that will only contain the metadata. > > Will ovirt support this in 3.6? > Not explicitly and I never tried it with hosted-engine but I suspect it works as far it will report replica level = 3. By the way you can save disk space on the third host but you still need three hosts. > Kind regards, > > Jorick Astrego > > > > > Met vriendelijke groet, With kind regards, > > Jorick Astrego > > *Netbulae Virtualization Experts * > -- > Tel: 053 20 30 270 i...@netbulae.eu Staalsteden 4-3A KvK 08198180 Fax: > 053 20 30 271 www.netbulae.eu 7547 TA Enschede BTW NL821234584B01 > -- > > > ___ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > > ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] why replica 3
On Wed, Sep 9, 2015 at 10:14 AM, Richard Neuboeck wrote: > On 04.09.15 10:02, Simone Tiraboschi wrote: > > Is there a reason why it has to be exactly replica 3? > > > > > > To have a valid quorum having the system being able to decide witch is > > the right and safe copy avoiding an issue called split brain. > > Under certain circumstances/issues (network issue, hosts down or > > whatever could happen) the data on different replica could diverge: if > > you have two and just two different hosts that claim each other > that its > > copy is the right one there is no way to automatically take the right > > decision. Having three hosts and setting the quorum according to that > > solves/mitigates the issue. > > > Thanks for the explanation. I do understand the problem but since > I'm somewhat limited in my hardware options is there a way to > override this requirement? Meaning if I change the checks for > replica 3 in the installation scripts does something else fail on > the way? > I'm advising that it's not a safe configuration so it's not recommended for a production environment. Having said that, as far as I know it's enforced only in the setup script so tweaking it should be enough. Otherwise, if you have enough disk space, you can also have a different trick: you could create a replica 3 volume with 2 bricks from a single host. It's not a safe procedure at all cause you still have only 2 hosts, so it's basically just replica 2, and in case of split brain the host with two copies will win by configuration which is not always the right decision. > In my case coherence checks would come from outside the storage and > vm host setup and fencing would be applied appropriately. > Can I ask how? > I would very much appreciate it if the particulars of the storage > setup could be either selected from a list of possibilities or be > ignored and just a warning be issued that this setup is not recommended. > > Thanks! > Richard > > > -- > /dev/null > > ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] why replica 3
On 09/09/2015 10:14 AM, Richard Neuboeck wrote: > On 04.09.15 10:02, Simone Tiraboschi wrote: >> Is there a reason why it has to be exactly replica 3? >> >> >> To have a valid quorum having the system being able to decide witch is >> the right and safe copy avoiding an issue called split brain. >> Under certain circumstances/issues (network issue, hosts down or >> whatever could happen) the data on different replica could diverge: if >> you have two and just two different hosts that claim each other > that its >> copy is the right one there is no way to automatically take the right >> decision. Having three hosts and setting the quorum according to that >> solves/mitigates the issue. > > Thanks for the explanation. I do understand the problem but since > I'm somewhat limited in my hardware options is there a way to > override this requirement? Meaning if I change the checks for > replica 3 in the installation scripts does something else fail on > the way? > > In my case coherence checks would come from outside the storage and > vm host setup and fencing would be applied appropriately. > > I would very much appreciate it if the particulars of the storage > setup could be either selected from a list of possibilities or be > ignored and just a warning be issued that this setup is not recommended. > > Thanks! > Richard > > As a side question, in Glusterfs 3.7 there is an "AFR arbiter volume" (http://gluster.readthedocs.org/en/release-3.7.0-1/Features/afr-arbiter-volumes/) that will only contain the metadata. Will ovirt support this in 3.6? Kind regards, Jorick Astrego Met vriendelijke groet, With kind regards, Jorick Astrego Netbulae Virtualization Experts Tel: 053 20 30 270 i...@netbulae.euStaalsteden 4-3A KvK 08198180 Fax: 053 20 30 271 www.netbulae.eu 7547 TA Enschede BTW NL821234584B01 ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] why replica 3
On 04.09.15 10:02, Simone Tiraboschi wrote: > Is there a reason why it has to be exactly replica 3? > > > To have a valid quorum having the system being able to decide witch is > the right and safe copy avoiding an issue called split brain. > Under certain circumstances/issues (network issue, hosts down or > whatever could happen) the data on different replica could diverge: if > you have two and just two different hosts that claim each other that its > copy is the right one there is no way to automatically take the right > decision. Having three hosts and setting the quorum according to that > solves/mitigates the issue. Thanks for the explanation. I do understand the problem but since I'm somewhat limited in my hardware options is there a way to override this requirement? Meaning if I change the checks for replica 3 in the installation scripts does something else fail on the way? In my case coherence checks would come from outside the storage and vm host setup and fencing would be applied appropriately. I would very much appreciate it if the particulars of the storage setup could be either selected from a list of possibilities or be ignored and just a warning be issued that this setup is not recommended. Thanks! Richard -- /dev/null signature.asc Description: OpenPGP digital signature ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] why replica 3
On Fri, Sep 4, 2015 at 8:56 AM, Richard Neuboeck wrote: > Hi oVirt group, > > I'm currently trying to set up an oVirt 3.6 self hosed engine on a > replica 2 gluster volume. As you can imagine it fails since it expects a > replica 3 volume. > > Since this part is hard coded during the installation this could be > easily remedied but on the other hand I'm not familiar with the oVirt > setup so I'm not sure if I would be disturbing something far a head that > again is puzzled by the lack of a replica 3 volume. > > Is there a reason why it has to be exactly replica 3? > To have a valid quorum having the system being able to decide witch is the right and safe copy avoiding an issue called split brain. Under certain circumstances/issues (network issue, hosts down or whatever could happen) the data on different replica could diverge: if you have two and just two different hosts that claim each other that its copy is the right one there is no way to automatically take the right decision. Having three hosts and setting the quorum according to that solves/mitigates the issue. > Shouldn't the redundancy of the storage back end be the responsibility > of the admin and therefore the option to choose (at least replicated and > distributed replicated) open? > > I'm asking because the storage back and I'm using is kind of a 'big' > thing and it's hardware is put together to facilitate the use of replica > 2. Sure I could create multiple bricks on those two machines but other > than a negative performance impact I would gain nothing. Since the price > tag of one of those servers is quite high the likelihood of getting > another one is exactly 0. So I'm sure it makes sens to you that I would > rather choose my own replication level than having one predetermined. > > All the best > Richard > > > ___ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > > ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[ovirt-users] why replica 3
Hi oVirt group, I'm currently trying to set up an oVirt 3.6 self hosed engine on a replica 2 gluster volume. As you can imagine it fails since it expects a replica 3 volume. Since this part is hard coded during the installation this could be easily remedied but on the other hand I'm not familiar with the oVirt setup so I'm not sure if I would be disturbing something far a head that again is puzzled by the lack of a replica 3 volume. Is there a reason why it has to be exactly replica 3? Shouldn't the redundancy of the storage back end be the responsibility of the admin and therefore the option to choose (at least replicated and distributed replicated) open? I'm asking because the storage back and I'm using is kind of a 'big' thing and it's hardware is put together to facilitate the use of replica 2. Sure I could create multiple bricks on those two machines but other than a negative performance impact I would gain nothing. Since the price tag of one of those servers is quite high the likelihood of getting another one is exactly 0. So I'm sure it makes sens to you that I would rather choose my own replication level than having one predetermined. All the best Richard signature.asc Description: OpenPGP digital signature ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
