Re: [ovirt-users] Hardware for Hyperconverged oVirt: Gluster storage best practice

[Mahdi Adnan]

Hi,


4 SSDs in "distributed replica 2" volume for VM images, with additional 20 HDDs 
in another volume.

We had some minor XFS issues with the HDDs volume.

as for monitoring, standard snmp with few scripts to read smart report, we're 
still looking for a better way to monitor Gluster.

hardware is Cisco UCS C220.


We have another setup but not HC, and its equipped with 96 SSDs only.

No major issues so far.


--

Respectfully
Mahdi A. Mahdi


From: ov...@fateknollogee.com <ov...@fateknollogee.com>
Sent: Sunday, June 11, 2017 4:45:30 PM
To: Mahdi Adnan
Cc: Barak Korren; Yaniv Kaul; Ovirt Users
Subject: Re: [ovirt-users] Hardware for Hyperconverged oVirt: Gluster storage 
best practice

Mahdi,

Can you share some more detail on your hardware?
How many total SSDs?
Have you had any drive failures?
How do you monitor for failed drives?
Was it a problem replacing failed drives?

On 2017-06-11 02:21, Mahdi Adnan wrote:
> Hi,
>
> In our setup, we used each SSD as a standalone brick "no RAID" and
> created distributed replica with sharding.
>
> Also, we are NOT managing Gluster from ovirt.
>
> --
>
> Respectfully
> MAHDI A. MAHDI
>
> -
>
> FROM: users-boun...@ovirt.org <users-boun...@ovirt.org> on behalf of
> Barak Korren <bkor...@redhat.com>
> SENT: Sunday, June 11, 2017 11:20:45 AM
> TO: Yaniv Kaul
> CC: ov...@fateknollogee.com; Ovirt Users
> SUBJECT: Re: [ovirt-users] Hardware for Hyperconverged oVirt: Gluster
> storage best practice
>
> On 11 June 2017 at 11:08, Yaniv Kaul <yk...@redhat.com> wrote:
>>
>>> I will install the o/s for each node on a SATADOM.
>>> Since each node will have 6x SSD for gluster storage.
>>> Should this be software RAID, hardware RAID or no RAID?
>>
>> I'd reckon that you should prefer HW RAID on software RAID, and some
> RAID on
>> no RAID at all, but it really depends on your budget, performance,
> and your
>> availability requirements.
>>
>
> Not sure that is the best advice, given the use of Gluster+SSDs for
> hosting individual VMs.
>
> Typical software or hardware RAID systems are designed for use with
> spinning disks, and may not yield any better performance on SSDs. RAID
> is also not very good when I/O is highly scattered as it probably is
> when running multiple different VMs.
>
> So we are left with using RAID solely for availability. I think
> Gluster may already provide that, so adding additional software or
> hardware layers for RAID may just degrade performance without
> providing any tangible benefits.
>
> I think just defining each SSD as a single Gluster brick may provide
> the best performance for VMs, but my understanding of this is
> theoretical, so I leave it to the Gluster people to provide further
> insight.
>
> --
> Barak Korren
> RHV DevOps team , RHCE, RHCi
> Red Hat EMEA
> redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hardware for Hyperconverged oVirt: Gluster storage best practice

[Mahdi Adnan]

Hi,


In our setup, we used each SSD as a standalone brick "no RAID" and created 
distributed replica with sharding.

Also, we are NOT managing Gluster from ovirt.

--

Respectfully
Mahdi A. Mahdi


From: users-boun...@ovirt.org <users-boun...@ovirt.org> on behalf of Barak 
Korren <bkor...@redhat.com>
Sent: Sunday, June 11, 2017 11:20:45 AM
To: Yaniv Kaul
Cc: ov...@fateknollogee.com; Ovirt Users
Subject: Re: [ovirt-users] Hardware for Hyperconverged oVirt: Gluster storage 
best practice

On 11 June 2017 at 11:08, Yaniv Kaul <yk...@redhat.com> wrote:
>
>> I will install the o/s for each node on a SATADOM.
>> Since each node will have 6x SSD for gluster storage.
>> Should this be software RAID, hardware RAID or no RAID?
>
> I'd reckon that you should prefer HW RAID on software RAID, and some RAID on
> no RAID at all, but it really depends on your budget, performance, and your
> availability requirements.
>

Not sure that is the best advice, given the use of Gluster+SSDs for
hosting individual VMs.

Typical software or hardware RAID systems are designed for use with
spinning disks, and may not yield any better performance on SSDs. RAID
is also not very good when I/O is highly scattered as it probably is
when running multiple different VMs.

So we are left with using RAID solely for availability. I think
Gluster may already provide that, so adding additional software or
hardware layers for RAID may just degrade performance without
providing any tangible benefits.

I think just defining each SSD as a single Gluster brick may provide
the best performance for VMs, but my understanding of this is
theoretical, so I leave it to the Gluster people to provide further
insight.

--
Barak Korren
RHV DevOps team , RHCE, RHCi
Red Hat EMEA
redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hardware for Hyperconverged oVirt: Gluster storage best practice

[ovirt]

I think just defining each SSD as a single Gluster brick may provide
the best performance for VMs, but my understanding of this is
theoretical, so I leave it to the Gluster people to provide further
insight.


Pardon my lack of knowledge (I'm an oVirt/Gluster newbie).
I assume the SSD-to-single-Gluster brick can be done using gdeploy in 
oVirt?


On 2017-06-11 01:20, Barak Korren wrote:

On 11 June 2017 at 11:08, Yaniv Kaul  wrote:



I will install the o/s for each node on a SATADOM.
Since each node will have 6x SSD for gluster storage.
Should this be software RAID, hardware RAID or no RAID?


I'd reckon that you should prefer HW RAID on software RAID, and some 
RAID on
no RAID at all, but it really depends on your budget, performance, and 
your

availability requirements.



Not sure that is the best advice, given the use of Gluster+SSDs for
hosting individual VMs.

Typical software or hardware RAID systems are designed for use with
spinning disks, and may not yield any better performance on SSDs. RAID
is also not very good when I/O is highly scattered as it probably is
when running multiple different VMs.

So we are left with using RAID solely for availability. I think
Gluster may already provide that, so adding additional software or
hardware layers for RAID may just degrade performance without
providing any tangible benefits.

I think just defining each SSD as a single Gluster brick may provide
the best performance for VMs, but my understanding of this is
theoretical, so I leave it to the Gluster people to provide further
insight.

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hardware for Hyperconverged oVirt: Gluster storage best practice

[Karli Sjöberg]

Den 11 juni 2017 15:43 skrev ov...@fateknollogee.com:>> I think just defining each SSD as a single Gluster brick may provide>> the best performance for VMs, but my understanding of this is>> theoretical, so I leave it to the Gluster people to provide further >> insight.>> Barak, very interesting, I had never thought of doing it this way but your idea does make sense.I assume Gluster is able to tolerate drive failures in the array?I'm also interested in hearing what the Gluster folks think about your approach?Here's an interesting article I recently read about how to set it up, using ZFS as RAID:http://45drives.blogspot.se/2016/11/an-introduction-to-clustering-how-to.html?m=1/KOn 2017-06-11 01:20, Barak Korren wrote:> On 11 June 2017 at 11:08, Yaniv Kaul  wrote:>> >>> I will install the o/s for each node on a SATADOM.>>> Since each node will have 6x SSD for gluster storage.>>> Should this be software RAID, hardware RAID or no RAID?>> >> I'd reckon that you should prefer HW RAID on software RAID, and some >> RAID on>> no RAID at all, but it really depends on your budget, performance, and >> your>> availability requirements.>> > > Not sure that is the best advice, given the use of Gluster+SSDs for> hosting individual VMs.> > Typical software or hardware RAID systems are designed for use with> spinning disks, and may not yield any better performance on SSDs. RAID> is also not very good when I/O is highly scattered as it probably is> when running multiple different VMs.> > So we are left with using RAID solely for availability. I think> Gluster may already provide that, so adding additional software or> hardware layers for RAID may just degrade performance without> providing any tangible benefits.> > I think just defining each SSD as a single Gluster brick may provide> the best performance for VMs, but my understanding of this is> theoretical, so I leave it to the Gluster people to provide further> insight.___Users mailing listUsers@ovirt.orghttp://lists.ovirt.org/mailman/listinfo/users___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hardware for Hyperconverged oVirt: Gluster storage best practice

[ovirt]

Mahdi,

Can you share some more detail on your hardware?
How many total SSDs?
Have you had any drive failures?
How do you monitor for failed drives?
Was it a problem replacing failed drives?

On 2017-06-11 02:21, Mahdi Adnan wrote:

Hi,

In our setup, we used each SSD as a standalone brick "no RAID" and
created distributed replica with sharding.

Also, we are NOT managing Gluster from ovirt.

--

Respectfully
MAHDI A. MAHDI

-

FROM: users-boun...@ovirt.org <users-boun...@ovirt.org> on behalf of
Barak Korren <bkor...@redhat.com>
SENT: Sunday, June 11, 2017 11:20:45 AM
TO: Yaniv Kaul
CC: ov...@fateknollogee.com; Ovirt Users
SUBJECT: Re: [ovirt-users] Hardware for Hyperconverged oVirt: Gluster
storage best practice

On 11 June 2017 at 11:08, Yaniv Kaul <yk...@redhat.com> wrote:



I will install the o/s for each node on a SATADOM.
Since each node will have 6x SSD for gluster storage.
Should this be software RAID, hardware RAID or no RAID?


I'd reckon that you should prefer HW RAID on software RAID, and some

RAID on

no RAID at all, but it really depends on your budget, performance,

and your

availability requirements.



Not sure that is the best advice, given the use of Gluster+SSDs for
hosting individual VMs.

Typical software or hardware RAID systems are designed for use with
spinning disks, and may not yield any better performance on SSDs. RAID
is also not very good when I/O is highly scattered as it probably is
when running multiple different VMs.

So we are left with using RAID solely for availability. I think
Gluster may already provide that, so adding additional software or
hardware layers for RAID may just degrade performance without
providing any tangible benefits.

I think just defining each SSD as a single Gluster brick may provide
the best performance for VMs, but my understanding of this is
theoretical, so I leave it to the Gluster people to provide further
insight.

--
Barak Korren
RHV DevOps team , RHCE, RHCi
Red Hat EMEA
redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hardware for Hyperconverged oVirt: Gluster storage best practice

[ovirt]

I think just defining each SSD as a single Gluster brick may provide
the best performance for VMs, but my understanding of this is
theoretical, so I leave it to the Gluster people to provide further 
insight.




Barak, very interesting, I had never thought of doing it this way but 
your idea does make sense.


I assume Gluster is able to tolerate drive failures in the array?
I'm also interested in hearing what the Gluster folks think about your 
approach?




On 2017-06-11 01:20, Barak Korren wrote:

On 11 June 2017 at 11:08, Yaniv Kaul  wrote:



I will install the o/s for each node on a SATADOM.
Since each node will have 6x SSD for gluster storage.
Should this be software RAID, hardware RAID or no RAID?


I'd reckon that you should prefer HW RAID on software RAID, and some 
RAID on
no RAID at all, but it really depends on your budget, performance, and 
your

availability requirements.



Not sure that is the best advice, given the use of Gluster+SSDs for
hosting individual VMs.

Typical software or hardware RAID systems are designed for use with
spinning disks, and may not yield any better performance on SSDs. RAID
is also not very good when I/O is highly scattered as it probably is
when running multiple different VMs.

So we are left with using RAID solely for availability. I think
Gluster may already provide that, so adding additional software or
hardware layers for RAID may just degrade performance without
providing any tangible benefits.

I think just defining each SSD as a single Gluster brick may provide
the best performance for VMs, but my understanding of this is
theoretical, so I leave it to the Gluster people to provide further
insight.

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hardware for Hyperconverged oVirt: Gluster storage best practice

[ovirt]

Is that a hyper-converged setup of both oVirt and Gluster?
We usually do it in batches of 3 nodes.


Yes, it is for a HC setup of both oVirt & Gluster.


..it really depends on your budget,
performance, and your availability requirements.


I would like to enhance the performance.


Makes sense (I could not see the rear bays - might have missed them).
Will you be able to put some SSDs there for caching?

This the correct part # 
https://www.supermicro.com/products/chassis/2U/216/SC216BE26-R920LPB

In all the oVirt docs/videos, no one mentioned using SSD for caching?
Therefore I was not planning on caching.
I planned to install oVirt node on the rear SSD's



On 2017-06-11 01:08, Yaniv Kaul wrote:

On Sat, Jun 10, 2017 at 1:43 PM,  wrote:


Martin,

Looking to test oVirt on real hardware (aka no nesting)

Scenario # 1:
1x Supermicro 2027TR-HTRF 2U 4 node server


Is that a hyper-converged setup of both oVirt and Gluster?
We usually do it in batches of 3 nodes.


I will install the o/s for each node on a SATADOM.
Since each node will have 6x SSD for gluster storage.
Should this be software RAID, hardware RAID or no RAID?


I'd reckon that you should prefer HW RAID on software RAID, and some
RAID on no RAID at all, but it really depends on your budget,
performance, and your availability requirements.


Scenario # 2:
3x SuperMicro SC216E16-R1200LPB 2U server
Each server has 24x 2.5" bays (front) + 2x 2.5" bays (rear)
I will install the o/s on the drives using the rear bays (maybe RAID
1?)


Makes sense (I could not see the rear bays - might have missed them).
Will you be able to put some SSDs there for caching?


For Gluster, we will use the 24 front bays.
Should this be software RAID, hardware RAID or no RAID?


Same answer is above.
Y.


Thanks
Femi
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users [1]




Links:
--
[1] http://lists.ovirt.org/mailman/listinfo/users

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hardware for Hyperconverged oVirt: Gluster storage best practice

[Barak Korren]

On 11 June 2017 at 11:08, Yaniv Kaul  wrote:
>
>> I will install the o/s for each node on a SATADOM.
>> Since each node will have 6x SSD for gluster storage.
>> Should this be software RAID, hardware RAID or no RAID?
>
> I'd reckon that you should prefer HW RAID on software RAID, and some RAID on
> no RAID at all, but it really depends on your budget, performance, and your
> availability requirements.
>

Not sure that is the best advice, given the use of Gluster+SSDs for
hosting individual VMs.

Typical software or hardware RAID systems are designed for use with
spinning disks, and may not yield any better performance on SSDs. RAID
is also not very good when I/O is highly scattered as it probably is
when running multiple different VMs.

So we are left with using RAID solely for availability. I think
Gluster may already provide that, so adding additional software or
hardware layers for RAID may just degrade performance without
providing any tangible benefits.

I think just defining each SSD as a single Gluster brick may provide
the best performance for VMs, but my understanding of this is
theoretical, so I leave it to the Gluster people to provide further
insight.

-- 
Barak Korren
RHV DevOps team , RHCE, RHCi
Red Hat EMEA
redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hardware for Hyperconverged oVirt: Gluster storage best practice

[Yaniv Kaul]

On Sat, Jun 10, 2017 at 1:43 PM,  wrote:

> Martin,
>
> Looking to test oVirt on real hardware (aka no nesting)
>
> Scenario # 1:
> 1x Supermicro 2027TR-HTRF 2U 4 node server
>

Is that a hyper-converged setup of both oVirt and Gluster?
We usually do it in batches of 3 nodes.

I will install the o/s for each node on a SATADOM.
> Since each node will have 6x SSD for gluster storage.
> Should this be software RAID, hardware RAID or no RAID?
>

I'd reckon that you should prefer HW RAID on software RAID, and some RAID
on no RAID at all, but it really depends on your budget, performance, and
your availability requirements.


>
> Scenario # 2:
> 3x SuperMicro SC216E16-R1200LPB 2U server
> Each server has 24x 2.5" bays (front) + 2x 2.5" bays (rear)
> I will install the o/s on the drives using the rear bays (maybe RAID 1?)
>

Makes sense (I could not see the rear bays - might have missed them).
Will you be able to put some SSDs there for caching?


> For Gluster, we will use the 24 front bays.
> Should this be software RAID, hardware RAID or no RAID?
>

Same answer is above.
Y.


> Thanks
> Femi
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Hardware for Hyperconverged oVirt: Gluster storage best practice

[ovirt]

Martin,

Looking to test oVirt on real hardware (aka no nesting)

Scenario # 1:
1x Supermicro 2027TR-HTRF 2U 4 node server
I will install the o/s for each node on a SATADOM.
Since each node will have 6x SSD for gluster storage.
Should this be software RAID, hardware RAID or no RAID?

Scenario # 2:
3x SuperMicro SC216E16-R1200LPB 2U server
Each server has 24x 2.5" bays (front) + 2x 2.5" bays (rear)
I will install the o/s on the drives using the rear bays (maybe RAID 1?)
For Gluster, we will use the 24 front bays.
Should this be software RAID, hardware RAID or no RAID?

Thanks
Femi
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users