Robert Dunkley wrote:
What kind of bandwidth do other users see with TCP on qperf?
using qperf I get the following
rc_bw 1.95 GB/s
rc_bi_bw 3.9 GB/s
tcp_bw 995 MB/s - datagram mode mtu 2044
tcp_bw 1.15 GB/s - connected mode mtu 65520
my systems have ConnectX DDR HCA (25418) and PCIe gen2
From your rc_bw results it seems that your systems have gen1 PCIe, and
from the ibstat output
I see that you have the one port Sinai HCA (25204). The datagram mode
results with connectx
should be better since it supports TCP stateless offloads such as
checksum and LSO.
To change the mode of ipoib use
echo $mode > /sys/class/net/$dev/mode where $mode is connected or datagram
As for latency, from my experience with qperf it doesn't give the
optimized latency results, so for verbs based results
I suggest to use ib_write_lat which is provided with the perftest
package, for tcp latency results, you might want to use netperf with
TCP_RR and UDP_RR tests, eg:
netperf -H 10.10.0.90 -D 1, -l 10 -fM -t TCP_RR -- -D L,R -m 4 -M 4
netperf -H 10.10.0.90 -D 1, -l 10 -fM -t UDP_RR -- -m 4 -M 4
running these on my systems I see 25k (and above) transactions per
second which suggests an RTT/2 latency of about 20us (40us / 2) you saw
with qperf...
Or
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
