Hi,
I'm testing QoS on opensm.
I work with OFED-1.4-20081123-0600.tgz and opensm-3.2.4_20081122_c732c34.
I've set up qos-policy file, SL2VL and VLArbitration Table (all in
attachement).
Results still have wrong values.
I've launch opensm -Q /etc/ofa/opensm.conf and use qperf tools to test
QoS implementation :
bandwith test :
---------------
SL1 should have 33% of bandwidth (1:64) , SL2 sould have 66% of
bandwidth (2:128)
cmd server :
qperf -lp 19766 & qperf -lp 19764
cmd client :
qperf 192.168.0.3 -lp 19766 -sl 1 rc_rdma_write_bw > sl1.txt & qperf
192.168.0.3 -lp 19764 -sl 2 rc_rdma_write_bw > sl2.txt
results :
sl1.txt : rc_rdma_write_bw:
bw = 1.7 GB/sec
sl2.txt : rc_rdma_write_bw:
bw = 1.7 GB/sec
latency test :
--------------
cmd server:
qperf -lp 19766 & qperf -lp 19764
cmd client :
qperf 192.168.0.3 -lp 19766 -sl 1 rc_rdma_write_lat > sl1.txt & qperf
192.168.0.3 -lp 19764 -sl 2 rc_rdma_write_lat > sl2.txt
results :
sl1.txt : rc_rdma_write_lat
latency = 12.9 us
sl2.txt : rc_rdma_write_lat:
latency = 12.9 us
I've tested pc to pc without switch between both.
I've a Mellanox ConnectX card on each pc with following features :
ConnectX® IB QDR, ConnectX IB HCA/TCA IC, dual-port, QDR, PCIe 2.0
PCIe 2.0 5.0GT/s
Firmware has been updated with latest version 2.5.9
[]# ibstat
CA 'mlx4_0'
CA type: MT26428
Number of ports: 2
Firmware version: 2.5.900
Hardware version: a0
Node GUID: 0x0002c903000290aa
System image GUID: 0x0002c903000290ad
Capability mask: 0x0251086a
When I use smpquery, results are the following :
VLCap:...........................VL0-7
VLHighLimit:.....................0
VLArbHighCap:....................8
VLArbLowCap:.....................8
VLStallCount:....................0
So, my configuration (cf opensm.conf in attachement) doesn't correspond
to smpquery results.
I've tried to restart openibd on both pc. I've restarted opensm (stop
and start),
I'v tested "opensm -Q conf_file" but my configuration is always unset.
Did I miss something or is it a bug ?
Thanks for your help.
Céline Bourde.
MY CONFIGURATION IS THE FOLLOWING :
I've added "options mlx4_core enable_qos=1" in modprobe.conf to set on QoS
I've configured qos-policy file with following rules :
-----------------------
qos-levels
qos-level
name: DEFAULT
sl: 0
end-qos-level
qos-level
name: MPI
sl: 1
end-qos-level
qos-level
name: Lustre
sl: 2
end-qos-level
end-qos-levels
----------------------
my qos settings in /etc/ofa/opensm.conf
-----------------------
#
# QoS OPTIONS
#
# Enable QoS setup
qos TRUE
# QoS policy file to be used
qos_policy_file /etc/opensm/qos-policy.conf
# QoS default options
qos_max_vls 15
qos_high_limit 0
qos_vlarb_high
0:4,1:0,2:0,3:0,4:0,5:0,6:0,7:0,8:0,9:0,10:0,11:0,12:0,13:0,14:0
qos_vlarb_low
0:0,1:4,2:4,3:4,4:4,5:4,6:4,7:4,8:4,9:4,10:4,11:4,12:4,13:4,14:4
qos_sl2vl 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7
# QoS CA options
qos_ca_max_vls 15
qos_ca_high_limit 0
qos_ca_vlarb_high 0:0,1:0,2:0
qos_ca_vlarb_low 0:1,1:64,2:128
qos_ca_sl2vl 0,1,2,3,4,6,7,8,9,10,11,12,13,14,7,5
# QoS Switch external ports options
qos_swe_max_vls 15
qos_swe_high_limit 0
qos_swe_vlarb_high 0:0,1:0,2:0
qos_swe_vlarb_low 0:1,1:64,2:128
qos_swe_sl2vl 0,1,2,3,4,6,7,8,9,10,11,12,13,14,7,5
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
