Re: [ovirt-users] recommendations for best performance and reliability

[Sahina Bose]

On Mon, Nov 13, 2017 at 3:33 PM, Rudi Ahlers  wrote:

> Hi,
>
> Can someone please give me some pointers, what would be the best setup for
> performance and reliability?
>
> We have the following hardware setup:
>
> 3x Supermicro server with following features per server:
> 128GB RAM
> 4x 8TB SATA HDD
> 2x SSD drives (intel_ssdsc2ba400g4 - 400GB DC S3710)
> 2x 12 core CPU (Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
> Quad port 10Gbe Inter NIC
> 2x 10GB Cisco switches (to isolate storage network from LAN)
>
> One of the servers will be in another office, with a 600Mb wireless link
> for Disaster Recovery.
>

All 3 servers need to be on the same network with low latency between
servers. Gluster volume is used in replica 3 configuration in oVirt, and
data will be written to all 3 servers synchronously. If you have one of the
3 servers in a remote location, your writes will be slow and the storage
will seem to be unavailable to oVirt.

For Disaster recovery, you can use gluster's geo-replication feature. This
does need an additional server in the remote location. See
https://www.ovirt.org/develop/release-management/features/gluster/gluster-dr/


>
> What is recommended for the best setup in terms of redundancy and speed?
>
> I am guessing GlusterFS with a Distributed Striped Replicated Volume
> across 3 of the servers.
>


Replica 3 gluster volume with sharding turned on for the gluster volume. If
you use the deployment via Cockpit option, the volume is created with all
recommended options.

You can also manually set this by enabling the virt profile, like this
# gluster volume set  group virt


>
> For added performance I want to use the SSD drives, perhaps with dm-cache?
>
>

> Should I combine the 4x HDD's using LVM on each host node?
> What about RAID 6?
>

Gluster can work with either RAID or JBOD ( with replica 3, JBOD can be
used, as gluster has the redundancy built in on the other nodes). Users
often choose RAID to aggregate capacity from multiple disks as single
brick, and also as the brick rebuilt time is offloaded to RAID layer in
case of any of the hard disk failures. The answer is it depends on your
needs and the hardware you have.



>
>
>
> Virtual Machines will then reside on the oVirt Cluster and any one of the
> 3 host nodes can fail, or any single HDD can fail and all should still
> work, right/?
>
>
That's correct. With a replica 3 approach, the oVirt environment is
available even when one of the 3 hosts fail.


>
>
> --
> Kind Regards
> Rudi Ahlers
> Website: http://www.rudiahlers.co.za
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] recommendations for best performance and reliability

[Eduardo Mayoral]

Just curious. Would Gluster with a JBOD configuration make sense in this
scenario? Maybe with dm-cache on the SSDs?

In general, does a Gluster JBOD configuration make sense as a backend
for oVirt?

Eduardo Mayoral Jimeno (emayo...@arsys.es)
Administrador de sistemas. Departamento de Plataformas. Arsys internet.
+34 941 620 145 ext. 5153

On 13/11/17 12:47, FERNANDO FREDIANI wrote:
>
> Helli Rudi
>
> Nice specs.
>
> I wouldn't use GlusterFS for this setup with the third server in a
> different location. Just have this server as an Standalone and
> replicate the VMs there. You won't have real time replication, but
> much less hassle and probably to have constant failures, specially
> knowing you have a wireless link.
>
> For the SSDs I have been using bcache with success. Relatively simple
> to setup and pretty good performance.
>
> For your specs as you have 4 mechanical disks I would recommend you to
> have a RAID 5 between them (4 disks is my limit for RAID 5) and a RAID
> 0 made of SSDs for the bcache device. If the RAID 0 fails for any
> reason it will fall back directly to the mechanical disks and you can
> do maintenance on the Node doing live migration in order to replace
> the failed disks.
>
> However as you have you have 2 remaining server to create your cluster
> you may need to consider GlusterFS on the top of this RAID to have the
> replication and Highavaibility.
>
> Hope it helps.
>
> Fernando
>
>
> On 13/11/2017 08:03, Rudi Ahlers wrote:
>> Hi, 
>>
>> Can someone please give me some pointers, what would be the best
>> setup for performance and reliability?
>>
>> We have the following hardware setup:
>>
>> 3x Supermicro server with following features per server:
>> 128GB RAM
>> 4x 8TB SATA HDD
>> 2x SSD drives (intel_ssdsc2ba400g4 - 400GB DC S3710)
>> 2x 12 core CPU (Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
>> Quad port 10Gbe Inter NIC
>> 2x 10GB Cisco switches (to isolate storage network from LAN)
>>
>> One of the servers will be in another office, with a 600Mb wireless
>> link for Disaster Recovery. 
>>
>> What is recommended for the best setup in terms of redundancy and speed?
>>
>> I am guessing GlusterFS with a Distributed Striped Replicated Volume
>> across 3 of the servers. 
>>
>> For added performance I want to use the SSD drives, perhaps with
>> dm-cache?
>>
>> Should I combine the 4x HDD's using LVM on each host node?
>> What about RAID 6?
>>
>>
>>
>> Virtual Machines will then reside on the oVirt Cluster and any one of
>> the 3 host nodes can fail, or any single HDD can fail and all should
>> still work, right/? 
>>
>>
>>
>>
>> -- 
>> Kind Regards
>> Rudi Ahlers
>> Website: http://www.rudiahlers.co.za
>>
>>
>> ___
>> Users mailing list
>> Users@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>
>
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] recommendations for best performance and reliability

[FERNANDO FREDIANI]

Hello Rudi

If you have a 4th server it may work,but I am not knowledgeable about 
Gluster's GEOreplication. Perhaps someone else can advise.


With regards RAID 5 for 4 disks this is intended for capacity and as 
mentioned 4 disks is the maximum I would use for RAID5. As you have SSDs 
and intend to use a caching technique like bcache ou dm-cache this 
should cover the performance hit of RAID 5 write.
If you consider using RAID 6 in only 4 disks you are much better of 
using RAID 10 then as you will have double write performance and same 
capacity. I normally use RAID 10 or in ZFS configurations (not this 
case) multiple vdevs of RAID6.


For RAID in Linux I have been using mdraid. I know LVM does some Raid 
but I personally never done myself so can't advise.


Regards.
Fernando


On 13/11/2017 10:19, Rudi Ahlers wrote:

Hi Fernando,

Thanx.

I meant to say, the 4th server will be in another office. It's about 
3Km away and I was thinking of using Gluster's GEOreplication for this 
purpose.


I am not a fond user of RAID5 at all. But this raises the question: 
does RAID add any unnecessary overhead? I would rather run RAID 6 or 
RAI10.
And then, if RAID is the preferred way (over LVM?), as I don't have 
dedicated hardware RAID cards, would mdraid add any benefit?


On Mon, Nov 13, 2017 at 1:47 PM, FERNANDO FREDIANI 
> wrote:


Helli Rudi

Nice specs.

I wouldn't use GlusterFS for this setup with the third server in a
different location. Just have this server as an Standalone and
replicate the VMs there. You won't have real time replication, but
much less hassle and probably to have constant failures, specially
knowing you have a wireless link.

For the SSDs I have been using bcache with success. Relatively
simple to setup and pretty good performance.

For your specs as you have 4 mechanical disks I would recommend
you to have a RAID 5 between them (4 disks is my limit for RAID 5)
and a RAID 0 made of SSDs for the bcache device. If the RAID 0
fails for any reason it will fall back directly to the mechanical
disks and you can do maintenance on the Node doing live migration
in order to replace the failed disks.

However as you have you have 2 remaining server to create your
cluster you may need to consider GlusterFS on the top of this RAID
to have the replication and Highavaibility.

Hope it helps.

Fernando


On 13/11/2017 08:03, Rudi Ahlers wrote:

Hi,

Can someone please give me some pointers, what would be the best
setup for performance and reliability?

We have the following hardware setup:

3x Supermicro server with following features per server:
128GB RAM
4x 8TB SATA HDD
2x SSD drives (intel_ssdsc2ba400g4 - 400GB DC S3710)
2x 12 core CPU (Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
Quad port 10Gbe Inter NIC
2x 10GB Cisco switches (to isolate storage network from LAN)

One of the servers will be in another office, with a 600Mb
wireless link for Disaster Recovery.

What is recommended for the best setup in terms of redundancy and
speed?

I am guessing GlusterFS with a Distributed Striped Replicated
Volume across 3 of the servers.

For added performance I want to use the SSD drives, perhaps with
dm-cache?

Should I combine the 4x HDD's using LVM on each host node?
What about RAID 6?



Virtual Machines will then reside on the oVirt Cluster and any
one of the 3 host nodes can fail, or any single HDD can fail and
all should still work, right/?




-- 
Kind Regards

Rudi Ahlers
Website: http://www.rudiahlers.co.za


___
Users mailing list
Users@ovirt.org 
http://lists.ovirt.org/mailman/listinfo/users




___
Users mailing list
Users@ovirt.org 
http://lists.ovirt.org/mailman/listinfo/users





--
Kind Regards
Rudi Ahlers
Website: http://www.rudiahlers.co.za


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] recommendations for best performance and reliability

[FERNANDO FREDIANI]

Helli Rudi

Nice specs.

I wouldn't use GlusterFS for this setup with the third server in a 
different location. Just have this server as an Standalone and replicate 
the VMs there. You won't have real time replication, but much less 
hassle and probably to have constant failures, specially knowing you 
have a wireless link.


For the SSDs I have been using bcache with success. Relatively simple to 
setup and pretty good performance.


For your specs as you have 4 mechanical disks I would recommend you to 
have a RAID 5 between them (4 disks is my limit for RAID 5) and a RAID 0 
made of SSDs for the bcache device. If the RAID 0 fails for any reason 
it will fall back directly to the mechanical disks and you can do 
maintenance on the Node doing live migration in order to replace the 
failed disks.


However as you have you have 2 remaining server to create your cluster 
you may need to consider GlusterFS on the top of this RAID to have the 
replication and Highavaibility.


Hope it helps.

Fernando


On 13/11/2017 08:03, Rudi Ahlers wrote:

Hi,

Can someone please give me some pointers, what would be the best setup 
for performance and reliability?


We have the following hardware setup:

3x Supermicro server with following features per server:
128GB RAM
4x 8TB SATA HDD
2x SSD drives (intel_ssdsc2ba400g4 - 400GB DC S3710)
2x 12 core CPU (Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
Quad port 10Gbe Inter NIC
2x 10GB Cisco switches (to isolate storage network from LAN)

One of the servers will be in another office, with a 600Mb wireless 
link for Disaster Recovery.


What is recommended for the best setup in terms of redundancy and speed?

I am guessing GlusterFS with a Distributed Striped Replicated Volume 
across 3 of the servers.


For added performance I want to use the SSD drives, perhaps with dm-cache?

Should I combine the 4x HDD's using LVM on each host node?
What about RAID 6?



Virtual Machines will then reside on the oVirt Cluster and any one of 
the 3 host nodes can fail, or any single HDD can fail and all should 
still work, right/?





--
Kind Regards
Rudi Ahlers
Website: http://www.rudiahlers.co.za


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] recommendations for best performance and reliability

[Rudi Ahlers]

Hi,

Can someone please give me some pointers, what would be the best setup for
performance and reliability?

We have the following hardware setup:

3x Supermicro server with following features per server:
128GB RAM
4x 8TB SATA HDD
2x SSD drives (intel_ssdsc2ba400g4 - 400GB DC S3710)
2x 12 core CPU (Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
Quad port 10Gbe Inter NIC
2x 10GB Cisco switches (to isolate storage network from LAN)

One of the servers will be in another office, with a 600Mb wireless link
for Disaster Recovery.

What is recommended for the best setup in terms of redundancy and speed?

I am guessing GlusterFS with a Distributed Striped Replicated Volume across
3 of the servers.

For added performance I want to use the SSD drives, perhaps with dm-cache?

Should I combine the 4x HDD's using LVM on each host node?
What about RAID 6?



Virtual Machines will then reside on the oVirt Cluster and any one of the 3
host nodes can fail, or any single HDD can fail and all should still work,
right/?




-- 
Kind Regards
Rudi Ahlers
Website: http://www.rudiahlers.co.za
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users