Re: [ovirt-users] [Gluster-users] GlusterFS performance with only one drive per host?

2018-03-25 Thread Manoj Pillai
My take is that unless you have loads of data and are trying to optimize
for cost/TB, HDDs are probably not the right choice. This is particularly
true for random I/O workloads for which HDDs are really quite bad.

I'd recommend a recent gluster release, and some tuning because the default
settings are not optimized for performance. Some options to consider:
client.event-threads
server.event-threads
cluster.choose-local
performance.client-io-threads

You can toggle the last two and see what works for you. You'd probably need
to set event-threads to 4 or more. Ideally you'd tune some of the thread
pools based on observed bottlenecks in collected stats. top (top -bHd 10 >
top_threads.out.txt) is great for this. Using 6 small drives/bricks instead
of 3 is also a good idea to reduce likelihood of rpc bottlenecks.

There has been an effort to improve gluster performance over fast SSDs.
Hence the recommendation to try with a recent release. You can also check
in on some of the issues being worked on:
https://github.com/gluster/glusterfs/issues/412
https://github.com/gluster/glusterfs/issues/410

-- Manoj

On Sat, Mar 24, 2018 at 4:14 AM, Jayme <jay...@gmail.com> wrote:

> Do you feel that SSDs are worth the extra cost or am I better off using
> regular HDDs?  I'm looking for the best performance I can get with glusterFS
>
> On Fri, Mar 23, 2018 at 12:03 AM, Manoj Pillai <mpil...@redhat.com> wrote:
>
>>
>>
>> On Thu, Mar 22, 2018 at 3:31 PM, Sahina Bose <sab...@redhat.com> wrote:
>>
>>>
>>>
>>> On Mon, Mar 19, 2018 at 5:57 PM, Jayme <jay...@gmail.com> wrote:
>>>
>>>> I'm spec'ing a new oVirt build using three Dell R720's w/ 256GB.  I'm
>>>> considering storage options.  I don't have a requirement for high amounts
>>>> of storage, I have a little over 1TB to store but want some overhead so I'm
>>>> thinking 2TB of usable space would be sufficient.
>>>>
>>>> I've been doing some research on Micron 1100 2TB ssd's and they seem to
>>>> offer a lot of value for the money.  I'm considering using smaller cheaper
>>>> SSDs for boot drives and using one 2TB micron SSD in each host for a
>>>> glusterFS replica 3 setup (on the fence about using an arbiter, I like the
>>>> extra redundancy replicate 3 will give me).
>>>>
>>>> My question is, would I see a performance hit using only one drive in
>>>> each host with glusterFS or should I try to add more physical disks.  Such
>>>> as 6 1TB drives instead of 3 2TB drives?
>>>>
>>>
>> It is possible. With SSDs the rpc layer can become the bottleneck with
>> some workloads, especially if there are not enough connections out to the
>> server side. We had experimented with a multi-connection model for this
>> reason:  https://review.gluster.org/#/c/19133/.
>>
>> -- Manoj
>>
>>>
>>> [Adding gluster-users for inputs here]
>>>
>>>
>>>> Also one other question.  I've read that gluster can only be done in
>>>> groups of three.  Meaning you need 3, 6, or 9 hosts.  Is this true?  If I
>>>> had an operational replicate 3 glusterFS setup and wanted to add more
>>>> capacity I would have to add 3 more hosts, or is it possible for me to add
>>>> a 4th host in to the mix for extra processing power down the road?
>>>>
>>>
>>> In oVirt, we support replica 3 or replica 3 with arbiter (where one of
>>> the 3 bricks is a low storage arbiter brick). To expand storage, you would
>>> need to add in multiples of 3 bricks. However if you only want to expand
>>> compute capacity in your HC environment, you can add a 4th node.
>>>
>>>
>>>> Thanks!
>>>>
>>>>
>>>> ___
>>>> Users mailing list
>>>> Users@ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>
>>>>
>>>
>>> ___
>>> Gluster-users mailing list
>>> gluster-us...@gluster.org
>>> http://lists.gluster.org/mailman/listinfo/gluster-users
>>>
>>
>>
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


Re: [ovirt-users] [Gluster-users] GlusterFS performance with only one drive per host?

2018-03-25 Thread Manoj Pillai
On Thu, Mar 22, 2018 at 3:31 PM, Sahina Bose  wrote:

>
>
> On Mon, Mar 19, 2018 at 5:57 PM, Jayme  wrote:
>
>> I'm spec'ing a new oVirt build using three Dell R720's w/ 256GB.  I'm
>> considering storage options.  I don't have a requirement for high amounts
>> of storage, I have a little over 1TB to store but want some overhead so I'm
>> thinking 2TB of usable space would be sufficient.
>>
>> I've been doing some research on Micron 1100 2TB ssd's and they seem to
>> offer a lot of value for the money.  I'm considering using smaller cheaper
>> SSDs for boot drives and using one 2TB micron SSD in each host for a
>> glusterFS replica 3 setup (on the fence about using an arbiter, I like the
>> extra redundancy replicate 3 will give me).
>>
>> My question is, would I see a performance hit using only one drive in
>> each host with glusterFS or should I try to add more physical disks.  Such
>> as 6 1TB drives instead of 3 2TB drives?
>>
>
It is possible. With SSDs the rpc layer can become the bottleneck with some
workloads, especially if there are not enough connections out to the server
side. We had experimented with a multi-connection model for this reason:
https://review.gluster.org/#/c/19133/.

-- Manoj

>
> [Adding gluster-users for inputs here]
>
>
>> Also one other question.  I've read that gluster can only be done in
>> groups of three.  Meaning you need 3, 6, or 9 hosts.  Is this true?  If I
>> had an operational replicate 3 glusterFS setup and wanted to add more
>> capacity I would have to add 3 more hosts, or is it possible for me to add
>> a 4th host in to the mix for extra processing power down the road?
>>
>
> In oVirt, we support replica 3 or replica 3 with arbiter (where one of the
> 3 bricks is a low storage arbiter brick). To expand storage, you would need
> to add in multiples of 3 bricks. However if you only want to expand compute
> capacity in your HC environment, you can add a 4th node.
>
>
>> Thanks!
>>
>>
>> ___
>> Users mailing list
>> Users@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>
> ___
> Gluster-users mailing list
> gluster-us...@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users