Hello Or ,
I will check the issue now.
The latest git tree is available at
git://git.openfabrics.org/~shamoya/perftest.git ,
I've just checked it and it works.
Will get back to you soon
Ido
On 11/17/2010 1:30 PM, Or Gerlitz wrote:
Or Gerlitz wrote:
Now, Jack, using this patch,
I could get rdma_lat which doesn't use the rdma-cm, which means setting all the
low-level QP params
"by the hand" to produce the SAME result of 3.9us as with the rdma-cm, as you
can see its one liner
patch which uses higher MTU of 2048 vs the hard coded MTU of 256 used in the
code. This is quite counter
intuitive, for packets whose size is> 256, correct? is there any known issue
that can
explain that?! The SA is convinced that 2048 (0x84) is the best MTU for that
path, both nodes
have ConnectX DDR with firmware 2.7.0
running with mtu=256 vs mtu=2048 on various msg size, you can see this
phenomena in action
2.70us for 1k msg with LOWER mtu vs 3.54us with mtu=2048
# ib_send_lat -a -m 256 celery
------------------------------------------------------------------
Send Latency Test
Inline data is used up to 400 bytes message
Connection type : RC
local address: LID 0x04 QPN 0x4c004b PSN 0xf91b6e
remote address: LID 0x04 QPN 0x4c004a PSN 0xb9e19a
Mtu : 256
------------------------------------------------------------------
#bytes #iterations t_min[usec] t_max[usec] t_typical[usec]
2 1000 1.04 10.33 1.06
4 1000 1.03 9.80 1.07
8 1000 1.03 7.89 1.07
16 1000 1.05 7.81 1.09
32 1000 1.09 7.77 1.11
64 1000 1.18 7.99 1.21
128 1000 1.48 8.05 1.51
256 1000 2.06 8.67 2.19
512 1000 2.31 14.38 2.38
1024 1000 2.62 14.54 2.70
2048 1000 3.26 14.90 3.36
4096 1000 4.54 28.84 4.64
8192 1000 7.19 18.48 7.30
16384 1000 12.61 24.15 12.75
32768 1000 23.46 34.77 24.30
65536 1000 45.19 56.48 45.75
131072 1000 88.64 102.99 89.22
262144 1000 175.78 186.59 175.97
524288 1000 349.16 359.92 349.35
1048576 1000 695.80 706.90 696.22
2097152 1000 1389.41 1400.87 1389.95
4194304 1000 2777.21 2799.41 2778.08
8388608 1000 5691.77 5752.48 5737.45
------------------------------------------------------------------
All resources were Released successfully
# ib_send_lat -a -m 2048 celery
------------------------------------------------------------------
Send Latency Test
Inline data is used up to 400 bytes message
Connection type : RC
local address: LID 0x04 QPN 0x54004b PSN 0x59384f
remote address: LID 0x04 QPN 0x54004a PSN 0xcac60e
Mtu : 2048
------------------------------------------------------------------
#bytes #iterations t_min[usec] t_max[usec] t_typical[usec]
2 1000 1.02 10.66 1.06
4 1000 1.03 54.53 1.06
8 1000 1.04 36.84 1.06
16 1000 1.05 34.58 1.08
32 1000 1.08 40.70 1.11
64 1000 1.18 66.86 1.21
128 1000 1.48 38.76 1.51
256 1000 2.07 39.15 2.18
512 1000 2.58 45.90 2.67
1024 1000 3.46 38.07 3.54
2048 1000 5.22 40.55 5.31
4096 1000 6.53 39.17 6.61
8192 1000 9.16 38.85 9.29
16384 1000 14.52 50.14 14.99
32768 1000 25.54 51.20 25.64
65536 1000 47.18 59.76 47.30
131072 1000 90.87 96.92 90.99
262144 1000 178.22 183.27 178.37
524288 1000 352.93 358.01 353.13
1048576 1000 702.37 707.69 702.66
2097152 1000 1401.30 1411.63 1401.66
4194304 1000 2799.16 2845.59 2801.92
8388608 1000 5833.30 5911.70 5886.55
------------------------------------------------------------------
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
