<html><body>Everyone,
This is just a response to the issue of nfs vs glusterfs and the performance
for glsuter as I think some of the information may be useful here and has not
been discussed.
For the sake of clarity, I do not run infiniband..but I am running 10GB. My
normal production speeds sit around 600MB/s to 700MB/s with the native gluster
client.
My setup has 12 servers each with a single 24 disk sata raid 5 10TB brick.
Gluster setup of 6x2.
Before I settled on this setup I run extensive tests over about 6 weeks to
confirm my setup... in my case glusterfs native out performed NFS considerably
in aggretgate data xfers....I also found that the peak performance of the
glusterfs client in my setup was at about 12 servers. This distributed the
write and read loads very well....after 12 servers adding more produces
diminishing returns.
I bring this up as no one has been talking about how their brick setup may be
effecting the performance as well as the number of servers hosting
bricks....Putting multiple bricks on a single server does not increase the load
capacity anywhere near like adding another server with the additional brick.
The point I wanted to make was you need to look at all sides of your setup in
order to get the best performance...In my case this involved evaluating the
raid setup (tuning the size of blocks), the file system used for the brick on
the raid (and tuning it, specifically for the size and types of files be
manipulated), the memory and cpu in the servers, the network BW, and the client
access method. I had to look at all of these (and each of them had an impact
on the final performance numbers) before I found my best setup.....I do not
think you can just unilaterally dismiss the gluster setup until you have done a
COMPLETE analysis for how to best setup your environment.
Just sharing my thoughts as when I first was setting up gluster I thought I
could just install it and tweak the options and be good to go but once I
understood everything it is dependent on and addressed all of those options and
tuning as well....I significantly improved my overall performance to well over
what I was able to achieve with nfs.
Feel free to shoot me with comments or questions.
Bryan Washer
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Fernando Frediani (Qube)
Sent: Monday, September 10, 2012 8:14 AM
To: 'Stephan von Krawczynski'; 'Whit Blauvelt'
Cc: '[email protected]'; 'Brian Candler'
Subject: Re: [Gluster-users] Throughout over infiniband
Well, I would say there is a reason, if the Gluster client performed as
expected.
Using the Gluster client it should in theory access the file(s) directly from
the nodes where they reside and not having to go though a single node exporting
the NFS folder which would then have to gather the file.
Yes the NFS has all the caching stuff but if the Gluster client behaviour was
similar it should be able to get similar performance which doesn't seem to be
what has been resported.
I did tests myself using Gluster client and NFS and NFS got better performance
also and I believe this is due the caching.
Fernando
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Stephan von Krawczynski
Sent: 10 September 2012 13:57
To: Whit Blauvelt
Cc: [email protected]; Brian Candler
Subject: Re: [Gluster-users] Throughout over infiniband
On Mon, 10 Sep 2012 08:06:51 -0400
Whit Blauvelt <[email protected]> wrote:
> On Mon, Sep 10, 2012 at 11:13:11AM +0200, Stephan von Krawczynski wrote:
> > [...]
> > If you're lucky you reach something like 1/3 of the NFS performance.
> [Gluster NFS Client]
> Whit
There is a reason why one would switch from NFS to GlusterFS, and mostly it is
redundancy. If you start using a NFS-client type you cut yourself off the
"complete solution". As said elsewhere you can as well export GlusterFS via
kernel-nfs-server. But honestly, it is a patch. It would be better by far if
things are done right, native glusterfs client in kernel-space.
And remember, generally there should be no big difference between NFS and
GlusterFS with bricks spread over several networks - if it is done how it
should be, without userspace.
--
MfG,
Stephan
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
NOTICE: This email and any attachments may contain confidential and proprietary
information of NetSuite Inc. and is for the sole use of the intended recipient
for the stated purpose. Any improper use or distribution is prohibited. If you
are not the intended recipient, please notify the sender; do not review, copy
or distribute; and promptly delete or destroy all transmitted information.
Please note that all communications and information transmitted through this
email system may be monitored by NetSuite or its agents and that all incoming
email is automatically scanned by a third party spam and filtering service
</body></html>
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
