On 04/03/16 12:22, "Sahina Bose" <[email protected]> wrote:
>
>On 03/04/2016 02:14 AM, Pavel Gashev wrote:
>>
>> Unfortunately, oVirt doesn't support multi-node local storage clusters.
>> And Gluster/CEPH doesn't work well over 1G network. It looks like that
>> the only way to use oVirt in a three-node cluster is to share local
>> storages over NFS. At least it makes possible to migrate VMs and move
>> disks among hardware nodes.
>
>
>Do you know of reported problems with Gluster over 1Gb network? I think
>10Gb is recommended, but 1Gb can also be used for gluster.
>(We use it in our lab setup, and haven't encountered any issues so far
>but of course, the workload may be different - hence the question)
Let's calculate. If I have a three node replicated gluster volume, each block
writing on a node copies the block to the other two nodes. Thus, maximal write
performance can't be above 50MB/s. Even it's acceptable for my workload, things
get worse in failure recovering scenario. Gluster works with files. When a node
fails and then recovers (even it's just a plain reboot), gluster copies the
whole file over network if the file is changed during node outage. So if I have
a 100GB VM disk, and guest system has written a 512-byte block to the disk, the
whole 100GB will be copied during recovery. It might take 20 minutes for 100GB,
and 3 hours for 1TB. And network will be 100% busy during recovery, so VMs on
other nodes will wait for I/O most of time. In other words, a plain reboot of a
node would result in datacenter out of service for several hours.
Things might be better if you have a distributed+replicated gluster volume. It
requires at least six nodes. But things are still bad when you try to rebalance
the volume after adding new bricks, or when a node has really failed and
replaced.
Thus, 1GB network is ok for a lab, but it's not ok for production. IMHO.
_______________________________________________
Users mailing list
[email protected]
http://lists.ovirt.org/mailman/listinfo/users
