On 04/03/16 13:50, "Sahina Bose" <[email protected]> wrote: >On 03/04/2016 04:13 PM, Pavel Gashev wrote: >> On 04/03/16 12:22, "Sahina Bose" <[email protected]> wrote: >>> On 03/04/2016 02:14 AM, Pavel Gashev wrote: >>>> Unfortunately, oVirt doesn't support multi-node local storage clusters. >>>> And Gluster/CEPH doesn't work well over 1G network. It looks like that >>>> the only way to use oVirt in a three-node cluster is to share local >>>> storages over NFS. At least it makes possible to migrate VMs and move >>>> disks among hardware nodes. >>> >>> Do you know of reported problems with Gluster over 1Gb network? I think >>> 10Gb is recommended, but 1Gb can also be used for gluster. >>> (We use it in our lab setup, and haven't encountered any issues so far >>> but of course, the workload may be different - hence the question) >> Let's calculate. If I have a three node replicated gluster volume, each >> block writing on a node copies the block to the other two nodes. Thus, >> maximal write performance can't be above 50MB/s. Even it's acceptable for my >> workload, things get worse in failure recovering scenario. Gluster works >> with files. When a node fails and then recovers (even it's just a plain >> reboot), gluster copies the whole file over network if the file is changed >> during node outage. So if I have a 100GB VM disk, and guest system has >> written a 512-byte block to the disk, the whole 100GB will be copied during >> recovery. It might take 20 minutes for 100GB, and 3 hours for 1TB. And >> network will be 100% busy during recovery, so VMs on other nodes will wait >> for I/O most of time. In other words, a plain reboot of a node would result >> in datacenter out of service for several hours. >> >> Things might be better if you have a distributed+replicated gluster volume. >> It requires at least six nodes. But things are still bad when you try to >> rebalance the volume after adding new bricks, or when a node has really >> failed and replaced. >> >> Thus, 1GB network is ok for a lab, but it's not ok for production. IMHO. > >Most of the problems that you outline here - related to healing and >replacing are addressed with the sharding translator. Sharding breaks >the large image file into smaller files, so that the entire file does >not have to be copied. More details here - >http://blog.gluster.org/2015/12/introducing-shard-translator/
Sure, I meant the same by mentioning distributed+replicated volumes. Actually, distributed+striped+replicated - https://access.redhat.com/documentation/en-US/Red_Hat_Storage/2.0/html/Administration_Guide/sect-User_Guide-Setting_Volumes-Distributed_Striped_Replicated.html _______________________________________________ Users mailing list [email protected] http://lists.ovirt.org/mailman/listinfo/users
