Re: [ovirt-users] [Gluster-users] GlusterFS performance with only one drive per host?

2018-03-25 Thread Vlad Kopylov 
The bottleneck is definitely not the disk speed with glusterFS, no
point of using SSD for bricks what so ever

-v

On Thu, Mar 22, 2018 at 6:01 AM, Sahina Bose  wrote:
>
>
> On Mon, Mar 19, 2018 at 5:57 PM, Jayme  wrote:
>>
>> I'm spec'ing a new oVirt build using three Dell R720's w/ 256GB.  I'm
>> considering storage options.  I don't have a requirement for high amounts of
>> storage, I have a little over 1TB to store but want some overhead so I'm
>> thinking 2TB of usable space would be sufficient.
>>
>> I've been doing some research on Micron 1100 2TB ssd's and they seem to
>> offer a lot of value for the money.  I'm considering using smaller cheaper
>> SSDs for boot drives and using one 2TB micron SSD in each host for a
>> glusterFS replica 3 setup (on the fence about using an arbiter, I like the
>> extra redundancy replicate 3 will give me).
>>
>> My question is, would I see a performance hit using only one drive in each
>> host with glusterFS or should I try to add more physical disks.  Such as 6
>> 1TB drives instead of 3 2TB drives?
>
>
> [Adding gluster-users for inputs here]
>
>>
>> Also one other question.  I've read that gluster can only be done in
>> groups of three.  Meaning you need 3, 6, or 9 hosts.  Is this true?  If I
>> had an operational replicate 3 glusterFS setup and wanted to add more
>> capacity I would have to add 3 more hosts, or is it possible for me to add a
>> 4th host in to the mix for extra processing power down the road?
>
>
> In oVirt, we support replica 3 or replica 3 with arbiter (where one of the 3
> bricks is a low storage arbiter brick). To expand storage, you would need to
> add in multiples of 3 bricks. However if you only want to expand compute
> capacity in your HC environment, you can add a 4th node.
>
>>
>> Thanks!
>>
>>
>> ___
>> Users mailing list
>> Users@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>
>
> ___
> Gluster-users mailing list
> gluster-us...@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] [Gluster-users] GlusterFS performance with only one drive per host?

2018-03-25 Thread Manoj Pillai 
My take is that unless you have loads of data and are trying to optimize
for cost/TB, HDDs are probably not the right choice. This is particularly
true for random I/O workloads for which HDDs are really quite bad.

I'd recommend a recent gluster release, and some tuning because the default
settings are not optimized for performance. Some options to consider:
client.event-threads
server.event-threads
cluster.choose-local
performance.client-io-threads

You can toggle the last two and see what works for you. You'd probably need
to set event-threads to 4 or more. Ideally you'd tune some of the thread
pools based on observed bottlenecks in collected stats. top (top -bHd 10 >
top_threads.out.txt) is great for this. Using 6 small drives/bricks instead
of 3 is also a good idea to reduce likelihood of rpc bottlenecks.

There has been an effort to improve gluster performance over fast SSDs.
Hence the recommendation to try with a recent release. You can also check
in on some of the issues being worked on:
https://github.com/gluster/glusterfs/issues/412
https://github.com/gluster/glusterfs/issues/410

-- Manoj

On Sat, Mar 24, 2018 at 4:14 AM, Jayme  wrote:

> Do you feel that SSDs are worth the extra cost or am I better off using
> regular HDDs?  I'm looking for the best performance I can get with glusterFS
>
> On Fri, Mar 23, 2018 at 12:03 AM, Manoj Pillai  wrote:
>
>>
>>
>> On Thu, Mar 22, 2018 at 3:31 PM, Sahina Bose  wrote:
>>
>>>
>>>
>>> On Mon, Mar 19, 2018 at 5:57 PM, Jayme  wrote:
>>>
 I'm spec'ing a new oVirt build using three Dell R720's w/ 256GB.  I'm
 considering storage options.  I don't have a requirement for high amounts
 of storage, I have a little over 1TB to store but want some overhead so I'm
 thinking 2TB of usable space would be sufficient.

 I've been doing some research on Micron 1100 2TB ssd's and they seem to
 offer a lot of value for the money.  I'm considering using smaller cheaper
 SSDs for boot drives and using one 2TB micron SSD in each host for a
 glusterFS replica 3 setup (on the fence about using an arbiter, I like the
 extra redundancy replicate 3 will give me).

 My question is, would I see a performance hit using only one drive in
 each host with glusterFS or should I try to add more physical disks.  Such
 as 6 1TB drives instead of 3 2TB drives?

>>>
>> It is possible. With SSDs the rpc layer can become the bottleneck with
>> some workloads, especially if there are not enough connections out to the
>> server side. We had experimented with a multi-connection model for this
>> reason:  https://review.gluster.org/#/c/19133/.
>>
>> -- Manoj
>>
>>>
>>> [Adding gluster-users for inputs here]
>>>
>>>
 Also one other question.  I've read that gluster can only be done in
 groups of three.  Meaning you need 3, 6, or 9 hosts.  Is this true?  If I
 had an operational replicate 3 glusterFS setup and wanted to add more
 capacity I would have to add 3 more hosts, or is it possible for me to add
 a 4th host in to the mix for extra processing power down the road?

>>>
>>> In oVirt, we support replica 3 or replica 3 with arbiter (where one of
>>> the 3 bricks is a low storage arbiter brick). To expand storage, you would
>>> need to add in multiples of 3 bricks. However if you only want to expand
>>> compute capacity in your HC environment, you can add a 4th node.
>>>
>>>
 Thanks!


 ___
 Users mailing list
 Users@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/users


>>>
>>> ___
>>> Gluster-users mailing list
>>> gluster-us...@gluster.org
>>> http://lists.gluster.org/mailman/listinfo/gluster-users
>>>
>>
>>
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] [Gluster-users] GlusterFS performance with only one drive per host?

2018-03-25 Thread Manoj Pillai 
On Thu, Mar 22, 2018 at 3:31 PM, Sahina Bose  wrote:

>
>
> On Mon, Mar 19, 2018 at 5:57 PM, Jayme  wrote:
>
>> I'm spec'ing a new oVirt build using three Dell R720's w/ 256GB.  I'm
>> considering storage options.  I don't have a requirement for high amounts
>> of storage, I have a little over 1TB to store but want some overhead so I'm
>> thinking 2TB of usable space would be sufficient.
>>
>> I've been doing some research on Micron 1100 2TB ssd's and they seem to
>> offer a lot of value for the money.  I'm considering using smaller cheaper
>> SSDs for boot drives and using one 2TB micron SSD in each host for a
>> glusterFS replica 3 setup (on the fence about using an arbiter, I like the
>> extra redundancy replicate 3 will give me).
>>
>> My question is, would I see a performance hit using only one drive in
>> each host with glusterFS or should I try to add more physical disks.  Such
>> as 6 1TB drives instead of 3 2TB drives?
>>
>
It is possible. With SSDs the rpc layer can become the bottleneck with some
workloads, especially if there are not enough connections out to the server
side. We had experimented with a multi-connection model for this reason:
https://review.gluster.org/#/c/19133/.

-- Manoj

>
> [Adding gluster-users for inputs here]
>
>
>> Also one other question.  I've read that gluster can only be done in
>> groups of three.  Meaning you need 3, 6, or 9 hosts.  Is this true?  If I
>> had an operational replicate 3 glusterFS setup and wanted to add more
>> capacity I would have to add 3 more hosts, or is it possible for me to add
>> a 4th host in to the mix for extra processing power down the road?
>>
>
> In oVirt, we support replica 3 or replica 3 with arbiter (where one of the
> 3 bricks is a low storage arbiter brick). To expand storage, you would need
> to add in multiples of 3 bricks. However if you only want to expand compute
> capacity in your HC environment, you can add a 4th node.
>
>
>> Thanks!
>>
>>
>> ___
>> Users mailing list
>> Users@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>
> ___
> Gluster-users mailing list
> gluster-us...@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] [Gluster-users] GlusterFS performance with only one drive per host?

2018-03-24 Thread Alex K 
I would go with at least 4 HDDs per host in RAID 10. Then focus on network
performance where bottleneck usualy is for gluster.



On Sat, Mar 24, 2018, 00:44 Jayme  wrote:

> Do you feel that SSDs are worth the extra cost or am I better off using
> regular HDDs?  I'm looking for the best performance I can get with glusterFS
>
> On Fri, Mar 23, 2018 at 12:03 AM, Manoj Pillai  wrote:
>
>>
>>
>> On Thu, Mar 22, 2018 at 3:31 PM, Sahina Bose  wrote:
>>
>>>
>>>
>>> On Mon, Mar 19, 2018 at 5:57 PM, Jayme  wrote:
>>>
 I'm spec'ing a new oVirt build using three Dell R720's w/ 256GB.  I'm
 considering storage options.  I don't have a requirement for high amounts
 of storage, I have a little over 1TB to store but want some overhead so I'm
 thinking 2TB of usable space would be sufficient.

 I've been doing some research on Micron 1100 2TB ssd's and they seem to
 offer a lot of value for the money.  I'm considering using smaller cheaper
 SSDs for boot drives and using one 2TB micron SSD in each host for a
 glusterFS replica 3 setup (on the fence about using an arbiter, I like the
 extra redundancy replicate 3 will give me).

 My question is, would I see a performance hit using only one drive in
 each host with glusterFS or should I try to add more physical disks.  Such
 as 6 1TB drives instead of 3 2TB drives?

>>>
>> It is possible. With SSDs the rpc layer can become the bottleneck with
>> some workloads, especially if there are not enough connections out to the
>> server side. We had experimented with a multi-connection model for this
>> reason:  https://review.gluster.org/#/c/19133/.
>>
>> -- Manoj
>>
>>>
>>> [Adding gluster-users for inputs here]
>>>
>>>
 Also one other question.  I've read that gluster can only be done in
 groups of three.  Meaning you need 3, 6, or 9 hosts.  Is this true?  If I
 had an operational replicate 3 glusterFS setup and wanted to add more
 capacity I would have to add 3 more hosts, or is it possible for me to add
 a 4th host in to the mix for extra processing power down the road?

>>>
>>> In oVirt, we support replica 3 or replica 3 with arbiter (where one of
>>> the 3 bricks is a low storage arbiter brick). To expand storage, you would
>>> need to add in multiples of 3 bricks. However if you only want to expand
>>> compute capacity in your HC environment, you can add a 4th node.
>>>
>>>
 Thanks!


 ___
 Users mailing list
 Users@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/users


>>>
>>> ___
>>> Gluster-users mailing list
>>> gluster-us...@gluster.org
>>> http://lists.gluster.org/mailman/listinfo/gluster-users
>>>
>>
>>
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] [Gluster-users] GlusterFS performance with only one drive per host?

2018-03-23 Thread Jayme 
Do you feel that SSDs are worth the extra cost or am I better off using
regular HDDs?  I'm looking for the best performance I can get with glusterFS

On Fri, Mar 23, 2018 at 12:03 AM, Manoj Pillai  wrote:

>
>
> On Thu, Mar 22, 2018 at 3:31 PM, Sahina Bose  wrote:
>
>>
>>
>> On Mon, Mar 19, 2018 at 5:57 PM, Jayme  wrote:
>>
>>> I'm spec'ing a new oVirt build using three Dell R720's w/ 256GB.  I'm
>>> considering storage options.  I don't have a requirement for high amounts
>>> of storage, I have a little over 1TB to store but want some overhead so I'm
>>> thinking 2TB of usable space would be sufficient.
>>>
>>> I've been doing some research on Micron 1100 2TB ssd's and they seem to
>>> offer a lot of value for the money.  I'm considering using smaller cheaper
>>> SSDs for boot drives and using one 2TB micron SSD in each host for a
>>> glusterFS replica 3 setup (on the fence about using an arbiter, I like the
>>> extra redundancy replicate 3 will give me).
>>>
>>> My question is, would I see a performance hit using only one drive in
>>> each host with glusterFS or should I try to add more physical disks.  Such
>>> as 6 1TB drives instead of 3 2TB drives?
>>>
>>
> It is possible. With SSDs the rpc layer can become the bottleneck with
> some workloads, especially if there are not enough connections out to the
> server side. We had experimented with a multi-connection model for this
> reason:  https://review.gluster.org/#/c/19133/.
>
> -- Manoj
>
>>
>> [Adding gluster-users for inputs here]
>>
>>
>>> Also one other question.  I've read that gluster can only be done in
>>> groups of three.  Meaning you need 3, 6, or 9 hosts.  Is this true?  If I
>>> had an operational replicate 3 glusterFS setup and wanted to add more
>>> capacity I would have to add 3 more hosts, or is it possible for me to add
>>> a 4th host in to the mix for extra processing power down the road?
>>>
>>
>> In oVirt, we support replica 3 or replica 3 with arbiter (where one of
>> the 3 bricks is a low storage arbiter brick). To expand storage, you would
>> need to add in multiples of 3 bricks. However if you only want to expand
>> compute capacity in your HC environment, you can add a 4th node.
>>
>>
>>> Thanks!
>>>
>>>
>>> ___
>>> Users mailing list
>>> Users@ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
>>>
>>>
>>
>> ___
>> Gluster-users mailing list
>> gluster-us...@gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-users
>>
>
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users