It was this near endless range of possibilities via permutation of the parts
that originally attracted me to oVirt.
Being clearly a member of the original Lego generation I imagined how you could
simply add blocks of this and that to rebuild to something new fantastic...,
limitless gluster scaling and HCI, VDO dedup/compression, SSD caching, nested
virtualization, geo-replication, OVA support, it just had everything I might
want!
The truth is rather ugly from what I have gathered around here during almost
three years.
You don't mention it explicitly, but you evidently are talking about a HCI
setup, preferably installed, expanded and managed by the nice Cockpit/Engine
GUIs.
What I learned is that Gluster based HCI receives very little love from the
oVirt developers, even if it seems to you and me (Lego people?) the most
attractive option.
My impression is that they tend to work more with the original non-HCI approach
based on SAN or NFS storage, even if the engine may now by default be a VM,
when it was separate servers originally.
Gluster, VDO, Ansible, Java engine are all aquisitions, HCI more of a
management decision to counter Nutanix and I find that following the evolution
from Moshe Bar's Mosix to Qumranet and the Permabit, Ansible and Gluster
acquisitions as well as the competitor products helps understand why things are
as they are.
The current oVirt code base supports single node HCI, which delivers no
benefits beyond testing. It also supports 3 node HCI, which kind of works with
the Cockpit wizard (I have never succeeded with an installation where I didn't
have to do some fiddling).
Beyond that you already branch out into the jungle of not-tested,
not-supported, even if I remember reading that 6-nodes and 9-node HCI seem
possible. But quorums with 6 nodes don't seem natural and certainly nobody
would want to use replicas in a 9-node HCI setup, right?
I've seem actual "not supported" comments in the oVirt Ansible code that stops
any installation using dispersed volumes, so there is your immediate answer.
I've tricked it past that point editing Ansible scripts and actually got oVirt
running with dispersed (erasure coded) volumes on 5 nodes, but it felt too
wobbly for real use. Instead I've added the CPU/RAM parts of these 5 nodes to a
3 node HCI setup and then used the disks to create an erasure coded Gluster,
which then can be used by VMs via Gluster or NFS pretty much like a filer.
I can't recommend that unless you're willing to pay the price of being on your
own.
One major issue here is that you can't just create glusters and then merge them
as any system can only ever be a member of one gluster. And you have to destroy
volumes before moving to an other gluster: not sure how that rhymes with
bottleneck-free scalability.
And then there are various parts in oVirt, which look for quorums and find them
missing even when nodes aren't actually contributing bricks to a volume
(compute-only hosts or non-HCI volumes).
I guess it's safe to say that everybody here would like to see you trying and
feeding the changes required to make it work back into the project...
...but it's not "supported out of the box".
P.S. When I'm offered erasure coding, VDO dedup/compression and thin
allocation, I naturally tend to tick all boxes (originally I also chose SSD
cache, but quickly changed to SSD-only storage). It's only later when they
mention somewhere, that you aren't supposed to use them in combination, without
a full explanation or analysis to follow. Because even at today's SSD storage
pricing I need all the tricks in the book I stayed with them all, but Gluster
doesn't seem to be a speed devil, no matter what you put on top.
And to think that there used to be a UDMA/Infiniband option, that later
disappeared with little comment...
VMs in a 9-node HCI replica gluster could only ever get ~1Gbit/s throughput on
a 10Gbit/s network, so erasure coding should be the smart choice...
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/WQ3LKYCOSKVGYRCCSFW7FW72HHWKUWXB/
