On Feb 4, 2011, at 1:15 PM, Conrad Wood wrote: > > On 4 Feb 2011, at 19:31, Doug Hughes wrote: > >> On 2/4/2011 1:11 PM, Conrad Wood wrote: >>> >>>> Re: 7gbits/etc. As others have said, you're over-running your CPU. IP >>>> over IB works just fine, but the packet sizes aren't matched up. What >>>> you really want to do is test it using actual native infiniband >>>> protocols. I recommend the Intel Benchmark Suite (IBM - nee Pallas). It >>>> will test infiniband typical MPI operations. >>> Fair enough, problem is, I am trying to provide highspeed internet access >>> to clients and they need tcp/ip (only). So I got to improve the speed of >>> IPoIB. >>> And yes, we have>20GBit/s internet connectivity. >>> >> parallel sessions on more than one CPU core. try iperf -P 4, or -P 8. >> if they have more than one session going simultaneously, they get this >> automatically (mostly) >> >> > > in my tests, parallel threads on the same machine did (in total) not exceed > 7GBit/s either. > there seems to be a single-threaded bottleneck somewhere. > Also, this won't help if clients do a single HTTP download. > So I am still trying to maximise TCP throughput. > I can (roughly) reproduce skylars finding so will work on the kernel code to > optimise performance and scale it across multiple cores as I reckon that's > where our bottleneck is.
Are you sure that the IB nic is plugged into a fully wired pci-e slot? This speed sounds suspiciously like what you could expect from a gen1 x4 pci-e slot after overhead. -nld _______________________________________________ Tech mailing list [email protected] https://lists.lopsa.org/cgi-bin/mailman/listinfo/tech This list provided by the League of Professional System Administrators http://lopsa.org/
