Hi, I'm trying to measure rdma write latencies. The platforms I'm using are a pair of 3GHz 64bit Xeons, 2GB ram, 2.6.13 vanilla kernel machines. The two machines are connected back-to-back through PCIe memfree DDR Mellanox HCA's (OpenSM handles the initialization of the "fabric"). When using the rdma_lat utility (gen2/trunk/src/userspace/perftest/rdma_lat/) to measure 4K rdma write latencies I get 22.8 usec (one way). Notice that I removed the IBV_SEND_INLINE flag from rdma_lat.c:376 in order to be able to send large non-inlined data. Also ".max_inline_data=size" was changed to ".max_inline_data=128" (line 342) otherwise the qp creation fails. When trying the perf_main utility from Mellanox 4.1 stack (vapi-linux-4_1_0.tgz) I get latencies of 12.8 usec. Is there a reason for the major latency differences? Do the two programs do rdma write latency tests in a fundamentally different way? Am I doing something wrong? The commands I used for rdma_lat are 10.100.1.130] ./rdma_lat 10.100.1.129] ./rdma_lat -s 4096 10.100.1.130 The commands for perf_main are: 10.100.1.129] perf_main --send --test=lat --rdma=write --size=4096 --iter=1000 -a10.100.1.130 10.100.1.130] perf_main --server Thanks, y
_______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
