Adrian,
I think you should really try to restart from scratch, i.e.
- setup ipoib with corresponding entries in DNS or /etc/hosts (make sure
the IB hosts are pingable)
- gluster peer probe on the ipoib hostnames
- recreate volume using ipoib hostnames for the bricks
For transport you can still use tcp (I usually set tcp,rdma to be
flexible later on)
I started naively myself a couple of weeks ago and did exactly the same
than you assuming that IB transport would be autodetected. In fact, the
100 MB/s you are seeing indicated very strongly that any communication
is still over Gigabit Ethernet (Which has a theoretical peak performance
of 125 MB/s but given switch quality and protocol overhead you will
reach more like 100-110 MB/s in reality)
Michael.
On 04/20/2012 07:42 PM, Bryan Whitehead wrote:
Max out the number of IO threads and apply a patch to make gluster
more agressive about spawning threads as in this thread:
http://gluster.org/pipermail/gluster-users/2012-February/009590.html
(the above thread is actually pretty good for getting performance out
of gluster with infiniband (I use IP over IB myself - could never get
the native rDMA to be fast enough).
On Fri, Apr 20, 2012 at 3:27 AM, Ionescu, A.<[email protected]> wrote:
I tried this, but the performance is still the same (about ~100MB/s).
A strange thing I noticed is that the node on which I run the benchmarking tool
reports ~350 MB/s traffic on the loopback interface.
Does anyone have any other suggestions?
Thanks,
Adrian
________________________________________
From: [email protected] [[email protected]] on
behalf of Sabuj Pattanayek [[email protected]]
Sent: 19 April 2012 22:32
To: [email protected]
Subject: Re: [Gluster-users] Performance issues with striped volume over
Infiniband
On Thu, Apr 19, 2012 at 2:00 PM, Ionescu, A.<[email protected]> wrote:
Thanks for your answer, Sabuj. However, I am not sure I understand what you
mean by trying with ipoib.
Do you mean specifying "transport tcp" and using the ipoib ips/hostnames for
the bricks?
If you manage to find a way to overcome this performance limitation, please
share with us your solution.
Yes.
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
