Arlin Davis wrote:
Chuck Hartley wrote:
We are doing performance measurements on an application that is using
uDAPL RDMA reads for some large transfers and the BW is less than we
expected. The transfers are 4MB and we are seeing BW of 930MiB/sec
(DDR). When we do the same transfer size using ib_read_bw we get 1475
MB/sec. On a pair of machines with SDR interfaces, we get 697MiB/sec
and 918MB/sec respectively.
Here is a quick comparison of verbs, rdma_cm, and uDAPL using
ib_read_bw, ib_write_bw, ib_rdma_bw -c, and dapltest. My
results are very close using default size of 65536.
1.) IB verbs: ib_read_bw, ib_write_bw
------------------------------------------------------------------
RDMA_Read BW Test
------------------------------------------------------------------
#bytes #iterations BW peak[MB/sec] BW average[MB/sec]
65536 10000 1331.10 1329.85
------------------------------------------------------------------
------------------------------------------------------------------
RDMA_Write BW Test
------------------------------------------------------------------
#bytes #iterations BW peak[MB/sec] BW average[MB/sec]
65536 10000 1423.43 1422.64
------------------------------------------------------------------
2.) RDMA_CM + verbs: ib_rdma_bw -c -n100000
10711: Bandwidth peak (#0 to #7603): 1428.73 MB/sec
10711: Bandwidth average: 1428.13 MB/sec
3.) DAPL + RDMA_CM + verbs:
dapltest -T P -m p -s cst-50-ib0 -i 10000 RW 65535
RDMA_WRITES:
Total Time : 4.55 sec
Total Data Exchanged : 6249.90 MB
CPU Utilization : 25.30
Operation Throughput : 21952.66 ops/sec
Bandwidth : 1372.2 MB/sec
dapltest -T P -m p -s cst-50-ib0 -i 10000 RR 65535
RDMA_READS
Total Time : 4.67 sec
Total Data Exchanged : 6249.90 MB
CPU Utilization : 25.26
Operation Throughput : 21384.77 ops/sec
Bandwidth : 1336.52 MB/sec
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
